HowTo make time machine backups on a samba fileserver without netatalk
They will use samba 4.7.4
In our environment we use a dedicated box only for time machine since it is possible with samba. So the normal file server is not affected and the updgrade process can be sloppy. :-)
HowTo make time machine backups on a samba fileserver without netatalk
Hi,
I just wanted to share my experience with the new vfs_fruit extension in samba 4.8
I've got three macbooks at home and want them to have time machine backups. The current way to go is a netatalk implementation, but it is horrible slow and buggy. But with the new vfs_fruit extension in samba 4.8 it works awesomely fine.
This copy+paste howto is just to get it running with some lazy workarounds and a little bit of documentation. Tested with Ubuntu 16.04
Some notes:
-
Samba 4.8 will not be in Ubuntu 18.04.
-
Currently Samba 4.8 is available as RC.
-
Release is currently planned for tomorrow, maybe (https://wiki.samba.org/index.php/Release_Planning_for_Samba_4.8)
-
Ensure you create encrypted backups
-
Tested with high sierra. Feel free to test it with other osx versions.
-
You can even make a full restore when you got a new macbook.
-
Don't blame me for anything. I am not even good at this stuff.
-
Feel free to message me with more input
-
I have very limited time to use reddit, so please be patient when I don't answer directly
Avahia:
apt -y install avahi-daemon
cat <<< _EOF_ > /etc/avahi/services/timemachine.service <?xml version="1.0" standalone='no'?> <!DOCTYPE service-group SYSTEM "avahi-service.dtd"> <service-group> <name replace-wildcards="yes">%h</name> <service> <type>_smb._tcp</type> <port>445</port> </service> <service> <type>_device-info._tcp</type> <port>0</port> <txt-record>model=RackMac</txt-record> </service> <service> <type>_adisk._tcp</type> <txt-record>sys=waMa=0,adVF=0x100</txt-record> <txt-record>dk0=adVN=TimeMachine Home,adVF=0x82</txt-record> </service> </service-group> _EOF_
There is the line <txt-record>dk0=adVN=TimeMachine Home,adVF=0x82</txt-record> where TimeMachine Home is the name your mac will display (https://imgur.com/a/iezWt).
Samba 4.8
prerequisites
apt -y install install acl attr autoconf bind9utils bison build-essential \ debhelper dnsutils docbook-xml docbook-xsl flex gdb libjansson-dev krb5-user \ libacl1-dev libaio-dev libarchive-dev libattr1-dev libblkid-dev libbsd-dev \ libcap-dev libcups2-dev libgnutls28-dev libgpgme11-dev libjson-perl \ libldap2-dev libncurses5-dev libpam0g-dev libparse-yapp-perl \ libpopt-dev libreadline-dev nettle-dev perl perl-modules pkg-config \ python-all-dev python-crypto python-dbg python-dev python-dnspython \ python3-dnspython python-gpgme python3-gpgme python-markdown python3-markdown \ python3-dev xsltproc zlib1g-dev
compiling
Get the current RC here: https://download.samba.org/pub/samba/rc/
wget https://download.samba.org/pub/samba/rc/samba-4.8.0rc4.tar.gz; tar -xzf samba-4.8.0rc4.tar.gz; cd samba-4.8.0rc4
./configure --with-systemd --systemd-install-services --with-systemddir=/lib/systemd/system \ --sysconfdir=/etc/samba --jobs=`nproc --all` make --jobs=`nproc --all` && make install
This will install samba to /usr/local/samba, puts configs under /etc/samba and installs the systemd services to /lib/systemd/system and it will use all your cpu cores for compiling.
Adjusting start scipt
sed -i 's/Type=notify/Type=simple/g' /lib/systemd/system/smb.service
Fixed some issues while starting the service.
configure samba
/etc/samba/smb.conf [global] # Basic Samba configuration server role = standalone server passdb backend = tdbsam obey pam restrictions = yes security = user printcap name = /dev/null load printers = no socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=524288 SO_SNDBUF=524288 server string = Samba Server %v map to guest = bad user dns proxy = no wide links = yes follow symlinks = yes unix extensions = no acl allow execute always = yes log file = /var/log/samba/%m.log max log size = 1000 # Special configuration for Apple's Time Machine fruit:model = MacPro fruit:advertise_fullsync = true fruit:aapl = yes ## Definde your shares here [TimeMachine Home] path = /srv/backup/timemachine/%U valid users = %U writable = yes durable handles = yes kernel oplocks = no kernel share modes = no posix locking = no vfs objects = catia fruit streams_xattr ea support = yes browseable = yes read only = No inherit acls = yes fruit:time machine = yes
Backups will be stores under /srv/backup/timemachine/$USERNAME
You will need to create the folders and the user:
mkdir -p /srv/backup/timemachine/
/usr/local/samba/bin/smbpasswd -a $USERNAME
mkdir -m 700 /srv/backup/timemachine/$USERNAME; chown $USERNAME /srv/backup/timemachine/$USERNAME
Quotas
To set quotas you need to put a .com.apple.TimeMachine.quota.plist file:
cat <<< _EOF_ > /srv/backup/timemachine/$USERNAME/.com.apple.TimeMachine.quota.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>GlobalQuota</key>
<integer>300000000000</integer>
</dict>
</plist>
_EOF_To calculate the interger use MB * 1000000 (300GB in this example).
starting
systemctl enable smb.service; systemctl start smb.service
done
Edit1) Some input from u/g-a-c
Hi,
I just wanted to share my experience with the new vfs_fruit extension in samba 4.8
I've got three macbooks at home and want them to have time machine backups. The current way to go is a netatalk implementation, but it is horrible slow and buggy. But with the new vfs_fruit extension in samba 4.8 it works awesomely fine.
This copy+paste howto is just to get it running with some lazy workarounds and a little bit of documentation. Tested with Ubuntu 16.04
Some notes:
-
Samba 4.8 will not be in Ubuntu 18.04.
-
Currently Samba 4.8 is available as RC.
-
Release is currently planned for tomorrow, maybe (https://wiki.samba.org/index.php/Release_Planning_for_Samba_4.8)
-
Ensure you create encrypted backups
-
Tested with high sierra. Feel free to test it with other osx versions.
-
You can even make a full restore when you got a new macbook.
-
Don't blame me for anything. I am not even good at this stuff.
-
Feel free to message me with more input
-
I have very limited time to use reddit, so please be patient when I don't answer directly
Avahia:
apt -y install avahi-daemon
cat <<< _EOF_ > /etc/avahi/services/timemachine.service <?xml version="1.0" standalone='no'?> <!DOCTYPE service-group SYSTEM "avahi-service.dtd"> <service-group> <name replace-wildcards="yes">%h</name> <service> <type>_smb._tcp</type> <port>445</port> </service> <service> <type>_device-info._tcp</type> <port>0</port> <txt-record>model=RackMac</txt-record> </service> <service> <type>_adisk._tcp</type> <txt-record>sys=waMa=0,adVF=0x100</txt-record> <txt-record>dk0=adVN=TimeMachine Home,adVF=0x82</txt-record> </service> </service-group> _EOF_
There is the line <txt-record>dk0=adVN=TimeMachine Home,adVF=0x82</txt-record> where TimeMachine Home is the name your mac will display (https://imgur.com/a/iezWt).
Samba 4.8
prerequisites
apt -y install install acl attr autoconf bind9utils bison build-essential \ debhelper dnsutils docbook-xml docbook-xsl flex gdb libjansson-dev krb5-user \ libacl1-dev libaio-dev libarchive-dev libattr1-dev libblkid-dev libbsd-dev \ libcap-dev libcups2-dev libgnutls28-dev libgpgme11-dev libjson-perl \ libldap2-dev libncurses5-dev libpam0g-dev libparse-yapp-perl \ libpopt-dev libreadline-dev nettle-dev perl perl-modules pkg-config \ python-all-dev python-crypto python-dbg python-dev python-dnspython \ python3-dnspython python-gpgme python3-gpgme python-markdown python3-markdown \ python3-dev xsltproc zlib1g-dev
compiling
Get the current RC here: https://download.samba.org/pub/samba/rc/
wget https://download.samba.org/pub/samba/rc/samba-4.8.0rc4.tar.gz; tar -xzf samba-4.8.0rc4.tar.gz; cd samba-4.8.0rc4
./configure --with-systemd --systemd-install-services --with-systemddir=/lib/systemd/system \ --sysconfdir=/etc/samba --jobs=`nproc --all` make --jobs=`nproc --all` && make install
This will install samba to /usr/local/samba, puts configs under /etc/samba and installs the systemd services to /lib/systemd/system and it will use all your cpu cores for compiling.
Adjusting start scipt
sed -i 's/Type=notify/Type=simple/g' /lib/systemd/system/smb.service
Fixed some issues while starting the service.
configure samba
/etc/samba/smb.conf [global] # Basic Samba configuration server role = standalone server passdb backend = tdbsam obey pam restrictions = yes security = user printcap name = /dev/null load printers = no socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=524288 SO_SNDBUF=524288 server string = Samba Server %v map to guest = bad user dns proxy = no wide links = yes follow symlinks = yes unix extensions = no acl allow execute always = yes log file = /var/log/samba/%m.log max log size = 1000 # Special configuration for Apple's Time Machine fruit:model = MacPro fruit:advertise_fullsync = true fruit:aapl = yes ## Definde your shares here [TimeMachine Home] path = /srv/backup/timemachine/%U valid users = %U writable = yes durable handles = yes kernel oplocks = no kernel share modes = no posix locking = no vfs objects = catia fruit streams_xattr ea support = yes browseable = yes read only = No inherit acls = yes fruit:time machine = yes
Backups will be stores under /srv/backup/timemachine/$USERNAME
You will need to create the folders and the user:
mkdir -p /srv/backup/timemachine/
/usr/local/samba/bin/smbpasswd -a $USERNAME
mkdir -m 700 /srv/backup/timemachine/$USERNAME; chown $USERNAME /srv/backup/timemachine/$USERNAME
Quotas
To set quotas you need to put a .com.apple.TimeMachine.quota.plist file:
cat <<< _EOF_ > /srv/backup/timemachine/$USERNAME/.com.apple.TimeMachine.quota.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>GlobalQuota</key>
<integer>300000000000</integer>
</dict>
</plist>
_EOF_To calculate the interger use MB * 1000000 (300GB in this example).
starting
systemctl enable smb.service; systemctl start smb.service
done
Edit1) Some input from u/g-a-c
I encountered the same issue, having my Mac refusing to create the initial sparse bundle files on my samba server. After incorporating the hints on https://wiki.samba.org/index.php/Configure_Samba_to_Work_Better_with_Mac_OS_X it finally started the backup.
Although I can only assume, the following comment caught my attention: How to store OS X metadata: fruit:metadata = stream
You could try switching from netatalk to stream. Maybe using netatalk prevents time machine from storing its metadata.
Good luck, Kay
Thought I'd post some additional info for people. My currently working smb.conf is as follows. Adding fruit:metadata = stream worked for only a short while for me, so below are the results of hours of additional testing.
Note this is on Debian 10 (Buster) so things like password change command will be different on other distributions. Also note that the order of modules in the setting vfs objects = catia fruit streams_xattr is significant and important.
# NOTE: Whenever you modify this file you should run the command
# "testparm" to check that you have not made any basic syntactic
# errors.
[global]
workgroup = WORKGROUP
min protocol = SMB2
log file = /var/log/samba/log.%m
max log size = 5000
logging = file
security = USER
panic action = /usr/share/samba/panic-action %d
server role = standalone server
obey pam restrictions = yes
unix password sync = yes
passwd program = /usr/bin/passwd %u
passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
pam password change = yes
username map = /etc/samba/users.map
map to guest = bad user
guest account = XXXYOURGUESTACCOUNT
# Time Machine settings
vfs objects = catia fruit streams_xattr
fruit:model = RackMac
fruit:advertise_fullsync = true
fruit:metadata = stream
fruit:veto_appledouble = no
#default is yes, not necessary to specify fruit:posix_rename = no
#default is yes, not necessary to specify fruit:zero_file_id = yes
fruit:wipe_intentionally_left_blank_rfork = yes
fruit:delete_empty_adfiles = yes
ea support = yes
#default is yes, not necessary to specify fruit:aapl = yes
# Make share visible to Windows
#Disabled for a Mac-Only network
# lanman auth = no
# ntlm auth = yes
# wins support = yes
# local master = yes
# preferred master = yes
# Allow symlinks
# follow symlinks = yes
# wide links = yes
# unix extensions = no
#======================= Share Definitions =======================
[SHARED DRIVE]
path = /PATH/TO/YOUR/SHARED/FOLDER
valid users = @YOURGUESTUSERGROUP
writable = yes
durable handles = yes
kernel oplocks = no
kernel share modes = no
posix locking = no
vfs objects = catia fruit streams_xattr
#default is yes, not necessary to specify ea support = yes
#default is yes, not necessary to specify browseable = yes
read only = no
inherit acls = yes
fruit:time machine = yes
fruit:metadata = stream
fruit:locking = netatalk
guest ok = yes
[Time Machine]
path = /PATH/TO/YOUR/SHARED/FOLDER
valid users = @YOURGUESTUSERGROUP
writable = yes
durable handles = yes
kernel oplocks = no
kernel share modes = no
posix locking = no
vfs objects = catia fruit streams_xattr
#default is yes, not necessary to specify ea support = yes
browseable = no
read only = no
inherit acls = yes
fruit:time machine = yes
fruit:metadata = stream
fruit:locking = netatalk
fruit:time machine max size = 1.9T
**EDIT: Turns out Samba 4.9 on Debian 10 DOES automatically advertise TimeMachine destinations, as long as there are no Samba-related advertisements in /etc/avahi/services. My system wasn't advertising the TimeMachine share because I also had a manual smb share setup for avahi. When I disabled BOTH, Samba properly advertised both.
*** OLD SEE ABOVE Also note that The Samba build with Debian 10 currently does not have built-in support for advertising Time Machine shares. Normally specifying fruit:time machine = yes will trigger Samba to advertise a Time Machine destination via Avahi, but this specific support is not built with the Samba included in Debian 10. You'll need to configure a .service definition in /etc/avahi/services/. I created time_machine_advert.service in that folder. You'll need to restart avahi to be sure it gets applied with sudo systemctl restart avahi. I have two Time Machine destinations. Multiples can be advertised within a single file by simply duplicating the txt record and incrementing dkX, i.e. dk1, dk2 etc.
<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="yes">%h</name>
<service>
<type>_adisk._tcp</type>
<txt-record>sys=waMa=0,adVF=0x100</txt-record>
<txt-record>dk0=adVN=SAMBA SHARE NAME EXACTLY,adVF=0x82</txt-record>
<txt-record>dk1=adVN=SAMBA SHARE NAME #2 EXACTLY,adVF=0x82</txt-record>
</service>
<service>
<type>_smb._tcp</type>
<port>445</port>
</service>
</service-group>
There are some additional useful sites for Samba configuration with regards to Time Machine here:
Samba team's official guide to Time Machine configuration, although this alone didn't get a working config for me: https://wiki.samba.org/index.php/Configure_Samba_to_Work_Better_with_Mac_OS_X
smb.conf reference: https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html
vfs_fruit (the Apple compatibility module for Samba) reference https://www.samba.org/samba/docs/current/man-html/vfs_fruit.8.html
Ken Murphy's smb.conf on GitHub that finally pushed me over the edge and got my setup working: https://github.com/KenMurphy/SambaConfigs/blob/master/smb.conf
A general Samba on Debian guide: https://www.antoneliasson.se/journal/time-machine-compatible-samba-on-debian-buster/
I've just set up a clone of an Apple Time Capsule device via the well documented method of setting up a Samba share and advertising it with mDNS.
For the backup itself, it's working just fine. However, I have the icon appearing in the Finder, and it would be nice if double-clicking it could take me to the underlying Samba share. However, I'm not sure if this is possible.
In my case I'm using dnsd through systemd-resolved, so I have:
[Service] Name=Time Capsule Type=_adisk._tcp TxtText=dk0=adVN=Time Capsule,adVF=0x82 sys=waMa=0,adVF=0x100
and also
[Service]
Name={{ services.timeMachine.displayName }}
Type=_device-info._tcp
TxtText=model=TimeCapsule8,119Samba is set up as standard on port 445. So the question is whether I can customise at least the _adisk._tcp to point to the underlying Samba share for neatness.
Is it possible? If so, how?
Thanks in advance