Setting up multi-node HDP Cluster
March 26, 2013 Leave a comment
As you may be already aware Hortonworks announced support of the HADOOP cluster on Windows platform. In the project where we involved at the moment, we’ve got luck to play with the multi-node HDP cluster. Personally I felt as I am back in the beginning of the 90th with all the configurations and much stuff which should run from the command line. However, the HDP is really a lot of fun and easy to setup and run. Here is the list of things you may need to know in order to save time setting up the HDP in your environment:
1. Even if it sounds very obvious, follow the documentation. Software listed in the SW requirements list should be installed and the parameters like JAVA_HOME have to be configured.
2. Pay attention to the Firewall ports which have to be opened! HDP needs a lot of ports, in some cases I was opening the range instead of the single ports (like, 50000-60000 instead of all the single ports in the 50K range).
3. Leave only one network card and disable the rest that you have on your box.
4. Disable IPv6. You can do it by executing following command for each interface:
netsh interface teredo set state disable
netsh interface 6to4 set state disabled
netsh interface isatap set state disabled
5. In the clusterproperties.txt make sure that you specify path for setting up HDP, pay attention – path with subfolders is not allowed, no space in the folder name allowed. The documentation misleading here. Example of the correct string:
6. Same configuration file, #Hosts section – the FULL UNC path will not work! Use short names instead, Like bellow:
7. Edit hosts file and add nodes names with their IP addresses there
8. Now you can run installation. Example of the setup sting:
C:\Windows\system32>msiexec /i "C:\HDP_SETUP\hdp-1.1.0-160.winpkg.msi" /lv "C:\HDP_SETUP\installer.log" HDP_LAYOUT="c:\HDP_SETUP\clusterproperties.txt" HDP_DIR="C:\hdp" DESTROY_DATA="Yes"
You will need to run this script on every node of the cluster!
9. In case if setup filed, make sure you run uninstall command before trying to re-run installation process.
msiexec /x "C:\HDP_SETUP\hdp-1.1.0-160.winpkg.msi" /lv "C:\HDP_SETUP\installer.log" DESTROY_DATA="no"
10. Now, before running setup you will need to enable Remote Scripting on every node. Looks like it doesn’t work correctly. Therefore even if setup was successful, you will see in the error log that something like this:
CAQuietExec: WINPKG: rd /s /q G:\HadoopInstallFiles\HadoopPackages\hdp-1.1.0-winpkg\resources\hadoop-1.1.0-SNAPSHOT.winpkg
CAQuietExec: powershell.exe : powershell.exe : powershell.exe : Stop-Service : Cannot find
CAQuietExec: any service with
CAQuietExec: At G:\HadoopInstallFiles\HadoopSetupTools\winpkg.ps1:118 char:9
CAQuietExec: + $out = powershell.exe -InputFormat none -Command "$command" 2>&1
CAQuietExec: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CAQuietExec: + CategoryInfo : NotSpecified: (powershell.exe …y service with
CAQuietExec: :String) , RemoteException
CAQuietExec: + FullyQualifiedErrorId : NativeCommandError
11. On every node manually start the services by calling start_local_hdp_services.cmd
12. If everything is done right, you will be able to run a smoke test with no errors and enjoy the HDP cluster