In Version 8 the Veeam Solutions Architect Team released a new format of the Best Practice Guide.
You can find the most actual version under:
It will be updated for v9 soon.
If you ever wanted to test or demo Veeam Backup & Replication functionallity without installing the product, now you can do this here: http://veeam.foonet.be/
my colleague and friend Tom Sightler created an toolset to backup SAP HANA with Veeam Backup & Replication. He documented everything in the Veeam Forum:
Basically it follows the same way that storage systems like NetApp use for Backup of HANA. You implement in Veeam Pre and Post Scripts that makes HANA aware of the Veeam Backups. As well Logfile Handling is included (how many backup data do you want to keep on HANA system itself?).
In case of a DB restore, you go to HANA Studio and can access the backup data on HANA system directly. If you need older versions you can restore them with Veeam File Level Recovery Wizard or more comfortable with the Veeam Enterprise Manager File Restore (Self Services) and hit the rescann button at HANA Studio restore wizard. They are detected and you can proceed with the restore.
sometimes ESXi NTP Service is a bit tricky. (Configuration see kb.vmware.com/kb/2012069)
When it do not update the time but all outputs show the correct NTP settings when you type in “watch ntpq” on ESXi console,
you can try to add the NTP Version to the /etc/ntp.conf .
“Server <NTP name or IP>”
“Server <NTP name or IP> version 3”
Specifically with Windows NTP Server you had to add this option to it.
Yes it is written at http://kb.vmware.com/kb/1005092
but It is at the end of the document hidden and in most cases people do the first steps in the document before they read the whole document and waste time. And… I didn´t found this solution at Google.
Update: There is as well a good KB that describe the Windows NTP + VMware ESXi configuration: http://kb.vmware.com/kb/1035833
as you might know Veeam do not install backup agents on the VMs to process application aware and application- and filesystem consistent backups. Veeam looks into the VM and it´s applications and register plus start an according run time environment that allow application aware backups.
We had lately an internal discussion about this topic and Anton Gostev Vice President of Product Management at Veeam Software allowed me to share his thoughts and ideas behind Veeam’s unique approach.
Andreas Neufert: “Let´s talk first about the definition of Agents. According to http://en.wikipedia.org/wiki/S
Anton Gostev: “All problems which cause issue known as “agent management hell” are brought by the persistency requirement
…(of that Agents from other solutions)…
– Need to constantly deploy agents to newly appearing VMs
– Need to update agents on all VMs
– Need to babysit agents on all VMs to ensure reliability (make sure it behaves correctly in the long run – memory leaks, conflicts with our software etc.)
Auto-injected temporary process addresses all of these issue, and the server stay clean of 3rd party code 99.9% of time.”
Andreas Neufert: “I think we all were at the point where we need to install a security patch in our application and have to wait till the backup vendor released a compatible backup agent version. Or I can remember that we have to boot all Servers because of a new version of such an agent (before I joined Veeam). But what happens if the Application Server/VM is down?”
Anton Gostev: “… Our architecture address the following two issues …
– Persistent agent (or in-guest process) requires VM from running at the time of backup in order to function. But no VMs are running 100% of time – some can be shutdown! We are equally impacted, however the major difference is that we do not REQUIRE that in-guest process was operating at the time of backup (all item-level recoveries are still possible, they just require a few extra steps). This is NOT the case with legacy agent-based architectures: shutdown VM means no item-level recoveries from the corresponding restore point.
– Legacy agent-based architectures require network connectivity from backup server to guest OS – rarely available, especially in secure or public cloud environments. We are not impacted, because we can failover to network-less interactions for our in-guest process. This is NOT the case with legacy agent-based architectures: for them it means no application-aware backup, and no item-level recoveries from the corresponding restore point.
Andreas Neufert: “Everyone who operate a DMZ knows the problem. You isolated the whole DMZ from your normal internal network, but the VMs need a network connection to the backup server which hold as well data from other systems. So the Veeam approach can bring additional security to the DMZ environment. Thank you Anton!”
Thanks for reading. Please send me comments if you want more interviews on this blog.
Update from 2019-05-20: Since some years the below SOAP modifications within vCenter are not needed anymore as Veeam caches all needed vCenter information in RAM which reduced the vCenter connection count drastically at the backup window. See Broker Service note here: https://helpcenter.veeam.com/docs/backup/vsphere/backup_server.html?ver=95u4
My friend and workmate Pascal Di Marco ran into some VMware connection limitation while backing up 4000VMs in a very short backup window.
If you ran a lot of parallel backup jobs that use the VMware VADP backup API you can run into 2 connection limitations… on vCenter SOAP connections and on some limitation on NFC buffer size on ESXi side.
All backup vendors that use VMware VADP implement in their product the VMware VDDK kit which help the backup vendor with some standard API calls and it also helps to read and write data. So all backup vendors have to deal with the VDDK own vCenter and ESXi connection count in addition to their own connections. VDDK connections vary from VDDK version to version.
So if you try to backup thousands of VMs in a very short time frames you can hit these limitations.
In case you hit that limitation, you can increase the vCenter SOAP connection limitation from 500 to 1000 by this VMware KB 2004663 http://kb.vmware.com/kb/2004663
EDIT: In vCenter Server 6.0, vpxd.cfg file is located at C:\ProgramData\VMware\vCenterServer\cfg\vmware-vpx
As well you can optimze the ESXI Network (NBD) performance by increasing the NFC buffer size from 16384 to 32768 MB and optimize the Cache Flush interval from 30s to 20s by VMware KB 2052302 http://kb.vmware.com/kb/2052302
on customer request I created a video that shows backup and single mail restore for lotus domino with Veeam Backup & Replication.
A Lotus Domino is non VSS aware (anyway this is the case under Linux). So you have only 2 options for consistent backups as IBM do not support VSS Filesystem only backups:
- Shutdown the VM => Service offline or at cluster do this only on one side.
- Close the connections and write the cache to disk
The question is why should I use a non Domino Backup API based backup?
For Veeam the answer is:
- Ultra Fast Serivce Restore with Instant VM Recovery (2min + OS boot)
- Easy to use Single Mail/Document restore
- Automated Restore Tests with SureBackup that test if a VM is Restoreable, OS boot, Network Connection is online and Domino Services are up and running on a daily base.
- And finally a backup on Image Level with Change Block Tracking based Incremental Forever is very efficient even at a Domino Server with high change rate.
Enjoy the video
there are some general tips and tricks for Backup & Replication that are not directly related with Veeam Software. I will update this blog post from time to time to share these tips.
1) Format Backup Target disks with “/l” to avoid that NTFS blocks access to your very large and frequently updated (fragmented) backup files.
format /FS:NTFS /L
This will take a while and will overwrite the selected folume (data loss be carefull).
If you have Win7/Win2008R2 you need to first install the following patch: http://support.microsoft.com/kb/967351/en-us
2) Fix CPU load VMXnet3 network card bug if you use one virtualized backup server/role or an VM with high disk load:
sometime the easiest things do not work and you have a tough time to find the root cause.
A customer of mine uses an virtual Windows 8.1 as a Veeam Backup Server and also as a VMware HotAdd Proxy for Branch Offices. For security reasons the UAC and Windows Firewall was enabled and username administrator disabled.
We found 3 major challenges in this situation:
– Backup & Replication was not able to run on an other local user than “Computername\Administrator” (“Can not access admin$ share” error.
– Random disconnect of VMware Tools with stunning Backup Jobs.
– After adding the Branch office Backup & Replication Server itself as a Veeam proxy to Head Quarters Backup & Replication server, local hot add processing was not possible anymore. Manual hotadd of disks was still possible… strange
Solutions for this situation:
– Enable File&Print sharing to use another local admin user than “Administrator.
– The second one was fixed by enabling “high performance” or “Höchstleistung” at the windows power options.
– Hotadd processing problem was related to different patch levels of B&R in the branch office. The HQs Backup & Replication Server was on a higher patch level and local branch office server was not able to process hotadd anymore. Running same patch level solved it.
Happy backup…. Andy
maybe this is an well known thing, but I never used an “!” at vSphere Password before and experienced some “inconvenience”. I installed vCenter Server and was not able to see the vcenter and also not able to access SSO configuration, because it was not there. (Remeber SSO configuration in 5.5 was rewritten and it is normaly found directly in vSphere Web Client – Administration tab.
“!” are not an allowed character at vSphere SSO Administrator password, but Setup process allow it.
If you used it in you password, you are able to logon to Web Client but you see no vSphere Server nor are you able to see the SSO configuration area. If you do not want to install your Server from scratch, you can use the following command to change the password:
Open SSH connection
The funny thing is, that you need to take care as well, that there is not an “!” in the auto generated password ;o)
After that login to Web Client and you can access SSO configuration now. In my case the vSphere Server showed up automatically after this as well.
Excellent article about VMFS locking. Everything what you need to know what VMFS on 5.0 do now and how VAAI is using it. Please check:
Check out this old ESX3.5 article about performance differences between vRDM, pRDM and VMDK. You can see that even in these good old ESX3.5 days there was no significant performance gap. As actual VMware Volumes do not have the 2TB limitation anymore, there is no real blocker to use VMDK (and vRDM).
If you use vRDM don´t forget to reserve some space next to the vmx file for snapshots (e.g. Backup-Snapshot-helper) :
in my Labs and presentations I find it very unhandy to typein connection and username/passwords at my vcenter client.
You can crete a link with parameter to do so.
Also you can change the UI language to your choice.
-locale Location (e.g. de-DE for GermanUI and en-US for Englisch UI)
-s für VCenter Server or ESX Host
“C:Program Files (x86)VMwareInfrastructureVirtual Infrastructure ClientLauncherVpxClient.exe” -u administrator -p XXX -s 192.168.1.1 -locale en-US
Maybe it is not a good idea to save your password for your production environment in a link, but it is very helpful for you lab environment. If you do not add the -p parameter, it will ask you for the password, but the rest of the settings are filled out