The regular Raspbian OS is 32-bit, and while this is fine for many light-weight uses, and even some CPU-heavy and IO-heavy uses, there is the problem with 32-bit file-size limitations in several utilities and programs.
Functions such as standard fseek() or ftell() deal with long integers for the locations, fgetpos() and fsetpos() also uses an internal offset type with is 4 bytes long.
To wit, running on a Pi3B with the standard Raspbian:
# include <stdio.h>
int main()
{
printf("Size of fpos_t = %zu\n", sizeof(fpos_t) );
printf("Size of off_t = %zu\n", sizeof(off_t) );
printf("Size of __off_t = %zu\n", sizeof( __off_t) );
printf("Size of __off64_t = %zu\n", sizeof( __off64_t) );
printf("Size of __mbstate_t = %zu\n", sizeof( __mbstate_t) );
return(0);
}
gives the information when run:
Size of fpos_t = 12 Size of off_t = 4 Size of __off_t = 4 Size of __off64_t = 8 Size of __mbstate_t = 8
From the file /usr/include/_G_config.h we see that the definition of fpos_t matches the 32-bit version, using the __off_t.
Similarly, the off_t used with lseek() is also 32-bit.
So even though the OS itself can handle files greater than 4GB, as evidenced by the succesful run of a dd command producing an 8G file full of zero-bytes:
pi@trident:~/src/spacet $ dd count=8192 if=/dev/zero of=nuller bs=1M 8192+0 records in 8192+0 records out 8589934592 bytes (8.6 GB) copied, 638.764 s, 13.4 MB/s pi@trident:~/src/spacet $ ls -al total 8388624 -rwxr-xr-x 1 pi pi 5908 Nov 6 14:23 fpz -rw-r--r-- 1 pi pi 336 Nov 6 14:23 fpz.c -rw-r--r-- 1 pi pi 8589934592 Nov 6 14:15 nuller pi@trident:~/src/spacet $
there is no guarantee that all other programs can. Thus the need for and utility of, a 64-bit system for file-storage purposes.
In the following, I detail the steps I took to make a system using a Pi3B+ and a portable hard drive run with Devuan 64-bits. While some readers may appreciate the absence of systemd as well, I've never had much problems with that. I chose Devuan primarily for its 64 bits availability.
The Pi3B+ version can boot directly from an USB drive without the need for any SD-card, and thus there is one less unreliable component that can fail. The Pi3B (non +, that is the earlier version) has to have some OTP bit set before it can boot from an USB drive, and this cannot be undone. I've decided to use a PI3B+ here, so I can boot from the drive right out of the gate. The hard disk is a Seagate 1TB portable drive.
The main Devuan page is at: https://devuan.org/, and from the download I selected the devuan.c3l.lu mirror (since it seemed reasonably close, geographically), then drill down into the FTP file tree, until I reached:
https://devuan.c3l.lu/devuan_ascii/embedded/
Here it was the matter of downloading
devuan_ascii_2.0.0_arm64_raspi3.img.xz
which is the 64-bit version for Raspberry Pi 3. There exists many other variants there for other processors.
Next, unpack the devuan_ascii_2.0.0_arm64_raspi3.img.xz using xz -d into devuan_ascii_2.0.0_arm64_raspi3.img. Then dd this to the USB disk:
dd if=devuan_ascii_2.0.0_arm64_raspi3.img bs=4M of=/dev/sdb
Of course, make sure the output disk is correct, it might be something else than /dev/sdb; use fdisk -l, maybe mount the disk and check its contents that is looks like it is the correct one, to make sure.
Obtain xz and unxz via apt install xz-utils if it isn't present.
It turned out, that the command-line and the file-system-table files, /boot/cmdline.txt and /etc/fstab, refer to the /dev/mmcblk0p2 and /dev/mmcblk0p1 block device names, rather than the /dev/sda2 and /dev/sda1 names.
Unless this is done, the startup never gets past the initial device discovery stages.
These have to be edited thus, after the drive has been written with the dd command above:
# mkdir r b # mount /dev/sdb1 b # mount /dev/sdb2 r # vi b/cmdline.txt
change /dev/mmcblk0p2 to /dev/sda2 for the rootdrive there
# vi r/etc/fstabchange
/dev/mmcblk0p1 /boot vfat defaults 0 1to
/dev/sda1 /boot vfat defaults 0 1 # umount r # umount b
Now the USB disk can be unplugged from the writer machine.
Connect the USB disk, the Pi3B+, to keyboard, display, and power, but not the network cable quite yet. The Pi should now boot the Devuan from the disk.
Login is root, password is toor. Obviously, the first thing to do now is to change this password to something not well-known. Having changed the password, now we can dare to plug in the ethernet cable.
The initial hostname is devuan, and now edit /etc/hosts and /etc/hostname to change this to whatever the desired hostname will be.
This initial system is very bare-bones, there is only the root user, and there isn't even any C compiler there. It also only has decided to occupy the first couple of GB on the disk, so another thing to do now will be to extend the root partition to some useful size, possibly the entire USB disk.
As the disk I connected is a spinning, 1TB one, (for file storage, remember), I may decide whether I want to have one great big partition or maybe two, a smaller root one and the largest one for payload. There may also be use for a swap partition, as the Pi only has 1GB RAM which can be a little tight for a 64-bit OS when doing lots of work.
If I wanted to do database or other large in-memory work, I'd stick with the 32-bit OSes as it will do a better job
I would (and have) use a 2GB PineA64 or a 4GB Rock64 for this kind of job.
For file-storage and file-serving, this will be just fine.
# fdisk /dev/sda list partitions, p to see the /dev/sda1 and /dev/sda2
we will delete and then re-create /dev/sda2, the second partition in vivo:
enter d for delete select partition 2 enter n for new partition enter p for primary partition then enter partition no 2 then accept the default start, this will be the same start point as before Now we can accept the default for the end which will be the entire disk, or we can leave a couple gigs at the end for a swap area, entering something like +928G. If fdisk asks for deleting an ext4 label say no to that then finally do w for write
Then shut down and re-start. Seems that the power system handling is a little off, reboot or shutdown -r turns the device off but doesn't turn it back on...
Notice that fdisk -l says that /dev/sda2 is 928G in size, yet df -h says that the root partition is only 1.7G. We haev to re-size the file system, this is simply:
# resize2fs /dev/sda2
After this, use fdisk again to add a third partition /dev/sda3 using the remainder of the disk, and set this to type 82, Linux Swap. This will be useful as swap space.
Edit /etc/fstab again, add a line for the swap device:
/dev/sda3 none swap sw 0 0
On next startup this will use this as swap space.
The procedure for extending the initial 1.7 GB root file-system to the whole SD-card if not using an external disk, is quite similar, referring to /dev/mmcblk0 instead of /dev/sda:
Start fdisk:
fdisk /dev/mmcblk0 list partitions, p to see the /dev/mmcblk0p1 and /dev/mmcblk0p2 enter d for delete select partition 2 enter n for new partition enter p for primary partition then enter partition no 2 then accept the default start, this will be the same start point as before Now we can accept the default for the end which will be the entire disk. If fdisk asks for deleting an ext4 label say no to that then finally do w for write
Now reboot
After reboot, the system still thinks the partition is only 1.7 M. We use the extend2fs /dev/mmcblk0p2 command to have it extend the space:
root@devuan:~# resize2fs /dev/mmcblk0p2 resize2fs 1.43.4 (31-Jan-2017) Filesystem at /dev/mmcblk0p2 is mounted on /; on-line resizing required old_desc_blocks = 1, new_desc_blocks = 4 The filesystem on /dev/mmcblk0p2 is now 7583488 (4k) blocks long. root@devuan:~#
fdisk -l now shows that this partition is the full available size.
Edit the /etc/passwd, /etc/shadow, /etc/group, /etc/gshadow to add one or more non-root users, whose home directories will be in /home, in the traditional way. Nothing very special here.
Can also use the adduser command, as it will deal with much of this automatically. Since I want to have the same uid and gid numbers on various systems, I'll want to use the --uid ## and --gid ## arguments.
As mentioned, this is a very lean system, containing just about the minimal number of tools for system maintenance. Notably there is no C compiler or other related tools in place.
First of all, update the packages list and the operating system files to the latest (and presumably greatest):
# apt update # apt upgrade
To add tools to make more compiled programs these are installed as follows.
# apt install gcc make build-essential
The latter will pull in additional required and useful packages, such as manpages-dev and also suggest a bunch of further additional packages, notably:
binutils-doc cpp-doc gcc-6-locales gcc-6-doc libstdc++6-6-dbg gcc-multilib autoconf automake libtool flex bison gdb gcc-doc libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan3-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx2-dbg libquadmath0-dbg glibc-doc libstdc++-6-doc make-doc man-browser
Apt install any or all of these as desired.
And we have a real 64-bit system at our service!
Size of void * = 8 Size of code * = 8 Size of size_t = 8 Size of time_t = 8 Size of long long = 8 Size of long = 8 Size of int = 4 Size of short = 2 Size of char = 1 Size of long double = 16 Size of double = 8 Size of float = 4
And looking at the same variables as we had up above, the C program shown, now tells us:
Size of fpos_t = 16 Size of off_t = 8 Size of __off_t = 8 Size of __off64_t = 8 Size of __mbstate_t = 8
So this shows, we now have complete support for all larger files.
I will also want to be able to use sqlite and zlib in my programs, so these will also have to be added:
apt install sqlite3 libsqlite3-dev apt install zlib1g-dev
To look for packages available, do apt list > aptliste.txt, then grep for interesting strings in that file.
apt install nfs-kernel-server nfs-common rpcbind
Then edit /etc/exports to include /home, /usr/local, or whatever else can be made available for NFS access.
exportfs -a /etc/init.d/nfs-kernel-server restart
apt install samba samba-common-bin
Add sections such as this in /etc/samba/smb.conf, a user named hware is shown:
[hware] comment = Hardware read only = no locking = no path = /home/hware guest ok = yes
Then chmod 1777 /home/hware
and add the SMB user: smbpasswd -a hware
testparm /etc/init.d/samba restart
There are no i2c-devices in place initially, but they can be enabled via device-tree overlays, by specifying things in /boot/config.txt. The /boot/overlays/README has lots of info on the details for all these. Actually, much of this is the same as found in Raspbian, but there is no equivalent to the raspi-config tool that masks all these details.
The relevant entries for standard hardware i2c busses, to turn them on, are, as follows:
For /dev/i2c-1 (SCL on GPIO2/P3 SCL on GPIO3/P5):
dtparam=i2c_arm=on
For /dev/i2c-0 (SCL on GPIO0/P27 SCL on GPIO1/P28):
dtparam=i2c_vc=on
We also need to add the i2c_dev in /etc/modules, so that the /dev/i2c-0 and /dev/i2c-1 show up at all.
i2c_dev
All these changes in /boot/config.txt require a reboot in order to take effect.
In order to access the i2c busses from command-line or programs, in the usual way we have to install these:
apt install libi2c-dev i2c-tools
There will be a group i2c added in /etc/group, and users of this group may access the i2c-devices. Also note that /usr/sbin/i2cdetect is the location of the i2cdetect program.
This one is quite straightforward:
dtparam=spi=on
Now /dev/spidev0.0 and /dev/spidev0.1 appear, and SPI runs is on the standard pins, see the GPIO maps . Note that initially these are only accessible by root. and there is no equivalent to the automatic group i2c for these, no automatic group spi.