Cloudera Administration ImportantNotice ©2010-2021Cloudera,Inc.Allrightsreserved. Cloudera,theClouderalogo,andanyotherproductor servicenamesorsloganscontainedinthisdocumentaretrademarksofClouderaand itssuppliersorlicensors,andmaynotbecopied,imitatedorused,inwholeorinpart, withoutthepriorwrittenpermissionofClouderaortheapplicabletrademarkholder.If thisdocumentationincludescode,includingbutnotlimitedto,codeexamples,Cloudera makesthisavailabletoyouunderthetermsoftheApacheLicense,Version2.0,including anyrequirednotices.AcopyoftheApacheLicenseVersion2.0,includinganynotices, isincludedherein.AcopyoftheApacheLicenseVersion2.0canalsobefoundhere: https://opensource.org/licenses/Apache-2.0 HadoopandtheHadoopelephantlogoaretrademarksoftheApacheSoftware Foundation.Allothertrademarks,registeredtrademarks,productnamesandcompany namesorlogosmentionedinthisdocumentarethepropertyoftheirrespectiveowners. Referencetoanyproducts,services,processesorotherinformation,bytradename, trademark,manufacturer,supplierorotherwisedoesnotconstituteorimply endorsement,sponsorshiporrecommendationthereofbyus. Complyingwithallapplicablecopyrightlawsistheresponsibilityoftheuser.Without limitingtherightsundercopyright,nopartofthisdocumentmaybereproduced,stored inorintroducedintoaretrievalsystem,ortransmittedinanyformorbyanymeans (electronic,mechanical,photocopying,recording,orotherwise),orforanypurpose, withouttheexpresswrittenpermissionofCloudera. Clouderamayhavepatents,patentapplications,trademarks,copyrights,orother intellectualpropertyrightscoveringsubjectmatterinthisdocument.Exceptasexpressly providedinanywrittenlicenseagreementfromCloudera,thefurnishingofthisdocument doesnotgiveyouanylicensetothesepatents,trademarkscopyrights,orother intellectualproperty.ForinformationaboutpatentscoveringClouderaproducts,see http://tiny.cloudera.com/patents. Theinformationinthisdocumentissubjecttochangewithoutnotice.Clouderashall notbeliableforanydamagesresultingfromtechnicalerrorsoromissionswhichmay bepresentinthisdocument,orfromuseofthisdocument. Cloudera,Inc. 395PageMillRoad PaloAlto,CA94306 [email protected] US:1-888-789-1488 Intl:1-650-362-0488 www.cloudera.com ReleaseInformation Version:ClouderaEnterprise5.5.x Date:February3,2021 Table of Contents About Cloudera Administration................................................................................7 Managing CDH and Managed Services.....................................................................8 ManagingCDHandManagedServicesUsingClouderaManager........................................................................8 Configuration Overview.........................................................................................................................................................8 Managing Clusters...............................................................................................................................................................32 ManagingServices...............................................................................................................................................................36 Managing Roles...................................................................................................................................................................45 ManagingHosts...................................................................................................................................................................49 MaintenanceMode..............................................................................................................................................................62 ManagingCDHUsingtheCommandLine..........................................................................................................65 StartingCDHServicesUsingtheCommandLine..................................................................................................................66 StoppingCDHServicesUsingtheCommandLine................................................................................................................71 MigratingDatabetweenClustersUsingdistcp....................................................................................................................73 ManagingIndividualServices.............................................................................................................................78 ManagingFlume..................................................................................................................................................................78 ManagingtheHBaseService...............................................................................................................................................80 Managing HDFS.................................................................................................................................................................134 ManagingHive...................................................................................................................................................................165 Managing Hue...................................................................................................................................................................173 Managing Impala..............................................................................................................................................................197 ManagingKey-ValueStoreIndexer....................................................................................................................................208 ManagingMapReduceandYARN......................................................................................................................................209 ManagingOozie.................................................................................................................................................................217 Managing Solr...................................................................................................................................................................225 ManagingSpark.................................................................................................................................................................228 ManagingtheSqoop1Client.............................................................................................................................................231 ManagingSqoop2.............................................................................................................................................................232 ManagingZooKeeper.........................................................................................................................................................232 ConfiguringServicestoUsetheGPLExtrasParcel.............................................................................................................233 Resource Management........................................................................................235 Schedulers........................................................................................................................................................235 Cloudera Manager Resource Management.....................................................................................................235 LinuxControlGroups(cgroups)........................................................................................................................237 ResourceManagementwithControlGroups.....................................................................................................................239 ConfiguringResourceParameters......................................................................................................................................240 StaticServicePools...........................................................................................................................................241 Dynamic Resource Pools..................................................................................................................................242 ManagingDynamicResourcePools...................................................................................................................................243 YARNPoolStatusandConfigurationOptions....................................................................................................................245 AssigningApplicationsandQueriestoResourcePools......................................................................................................246 Configuration Sets.............................................................................................................................................................248 Scheduling Rules................................................................................................................................................................249 Managing Impala Admission Control...............................................................................................................250 Managingthe Impala Llama ApplicationMaster..............................................................................................252 EnablingIntegratedResourceManagementUsingClouderaManager............................................................................253 DisablingIntegratedResourceManagementUsingClouderaManager...........................................................................254 ConfiguringLlamaUsingClouderaManager.....................................................................................................................254 Impala Resource Management........................................................................................................................254 AdmissionControlandQueryQueuing..............................................................................................................................254 IntegratedResourceManagementwithYARN...................................................................................................................262 Performance Management...................................................................................264 OptimizingPerformanceinCDH.......................................................................................................................264 ChoosingaDataCompressionFormat.............................................................................................................267 Tuningthe Solr Server......................................................................................................................................268 TuningtoCompleteDuringSetup......................................................................................................................................268 GeneralTuning...................................................................................................................................................................268 Other Resources.................................................................................................................................................................275 Tuning Spark Applications................................................................................................................................275 Tuning YARN.....................................................................................................................................................281 Overview............................................................................................................................................................................282 Cluster Configuration.........................................................................................................................................................285 YARN Configuration...........................................................................................................................................................286 MapReduce Configuration.................................................................................................................................................288 Step7:MapReduceConfiguration.....................................................................................................................................288 Step7A:MapReduceSanityChecking................................................................................................................................288 ConfiguringYourClusterInClouderaManager.................................................................................................................288 High Availability...................................................................................................290 HDFS High Availability......................................................................................................................................290 IntroductiontoHDFSHighAvailability...............................................................................................................................290 ConfiguringHardwareforHDFSHA...................................................................................................................................291 EnablingHDFSHA..............................................................................................................................................................292 DisablingandRedeployingHDFSHA..................................................................................................................................304 ConfiguringOtherCDHComponentstoUseHDFSHA.......................................................................................................305 AdministeringanHDFSHighAvailabilityCluster...............................................................................................................308 ChangingaNameserviceNameforHighlyAvailableHDFSUsingClouderaManager......................................................312 MapReduce(MRv1)andYARN(MRv2)HighAvailability..................................................................................312 YARN(MRv2)ResourceManagerHighAvailability.............................................................................................................313 WorkPreservingRecoveryforYARNComponents.............................................................................................................320 MapReduce(MRv1)JobTrackerHighAvailability..............................................................................................................322 ClouderaNavigatorKeyTrusteeServerHighAvailability.................................................................................334 ConfiguringKeyTrusteeServerHighAvailabilityUsingClouderaManager......................................................................334 ConfiguringKeyTrusteeServerHighAvailabilityUsingtheCommandLine......................................................................335 RecoveringaKeyTrusteeServer........................................................................................................................................337 KeyTrusteeKMSHighAvailability....................................................................................................................337 HighAvailabilityforOtherCDHComponents...................................................................................................338 HBaseHighAvailability......................................................................................................................................................338 Hive Metastore High Availability.......................................................................................................................................343 HueHighAvailability.........................................................................................................................................................345 Llama High Availability......................................................................................................................................................348 ConfiguringOozieforHighAvailability..............................................................................................................................349 Search High Availability.....................................................................................................................................................350 ConfiguringClouderaManagerforHighAvailabilityWithaLoadBalancer.....................................................352 IntroductiontoClouderaManagerDeploymentArchitecture...........................................................................................352 PrerequisitesforSettingupClouderaManagerHighAvailability......................................................................................353 High-LevelStepstoConfigureClouderaManagerHighAvailability.................................................................................354 DatabaseHighAvailabilityConfiguration..........................................................................................................................380 TLSandKerberosConfigurationforClouderaManagerHighAvailability.........................................................................381 Backup and Disaster Recovery..............................................................................383 PortRequirementsforBackupandDisasterRecovery.....................................................................................383 Data Replication...............................................................................................................................................384 Designating a Replication Source......................................................................................................................................386 HDFS Replication................................................................................................................................................................387 Hive Replication.................................................................................................................................................................396 Impala Metadata Replication............................................................................................................................................404 UsingSnapshotswithReplication......................................................................................................................................404 EnablingReplicationBetweenClustersinDifferentKerberosRealms................................................................................404 Replication of Encrypted Data...........................................................................................................................................406 HBase Replication..............................................................................................................................................................407 Snapshots.........................................................................................................................................................414 Cloudera Manager Snapshot Policies................................................................................................................................414 Managing HBase Snapshots..............................................................................................................................................417 ManagingHDFSSnapshots................................................................................................................................................428 Cloudera Manager Administration........................................................................433 Starting,Stopping,andRestartingtheClouderaManagerServer...................................................................433 ConfiguringClouderaManagerServerPorts....................................................................................................433 MovingtheClouderaManagerServertoaNewHost.....................................................................................433 ManagingtheClouderaManagerServerLog...................................................................................................434 Viewing the Log.................................................................................................................................................................434 SettingtheClouderaManagerServerLogLocation..........................................................................................................435 ClouderaManagerAgents................................................................................................................................435 Starting,Stopping,andRestartingClouderaManagerAgents..........................................................................................436 ConfiguringClouderaManagerAgents.............................................................................................................................437 ManagingClouderaManagerAgentLogs.........................................................................................................................440 ChangingHostnames........................................................................................................................................441 Configuring Network Settings..........................................................................................................................443 Alerts................................................................................................................................................................443 Managing Alerts................................................................................................................................................................444 Managing Licenses...........................................................................................................................................451 SendingUsageandDiagnosticDatatoCloudera.............................................................................................456 ConfiguringaProxyServer.................................................................................................................................................456 ManagingAnonymousUsageDataCollection..................................................................................................................456 ManagingHueAnalyticsDataCollection..........................................................................................................................456 DiagnosticDataCollection.................................................................................................................................................457 ExportingandImportingClouderaManagerConfiguration.............................................................................459 Backing up Cloudera Manager.........................................................................................................................459 BackingupDatabases........................................................................................................................................................461 OtherClouderaManagerTasksandSettings...................................................................................................461 Settings..............................................................................................................................................................................461 Alerts..................................................................................................................................................................................462 Users..................................................................................................................................................................................462 Kerberos.............................................................................................................................................................................462 License...............................................................................................................................................................................462 UserInterfaceLanguage....................................................................................................................................................462 Peers..................................................................................................................................................................................462 Cloudera Management Service........................................................................................................................462 Cloudera Navigator Data Management Component Administration......................468 ClouderaNavigatorAuditServer......................................................................................................................468 PublishingAuditEvents......................................................................................................................................................471 ClouderaNavigatorMetadataServer...............................................................................................................472 Managing Metadata Extraction........................................................................................................................................478 ManagingMetadataPolicies.............................................................................................................................................479 Appendix: Apache License, Version 2.0.................................................................480 AboutClouderaAdministration About Cloudera Administration ThisguidedescribeshowtoconfigureandadministeraClouderadeployment.Administratorsmanageresources, availability,andbackupandrecoveryconfigurations.Inaddition,thisguideshowshowtoimplementhighavailability, anddiscussesintegration. ClouderaAdministration|7 ManagingCDHandManagedServices Managing CDH and Managed Services IfyouuseClouderaManagertomanageyourcluster,configuringandmanagingyourcluster,aswellasindividual servicesandhosts,usesadifferentparadigmthanifyouuseCDHwithoutClouderaManager.Forthisreason,many oftheseconfigurationtasksoffertwodifferentsubtasks,oneeachforclustersmanagedbyClouderaManagerand oneforclusterswhichdonotuseClouderaManager.Often,thetasksarenotinterchangeable.Forinstance,ifyouuse ClouderaManageryoucannotusestandardHadoopcommand-lineutilitiestostartandstopservices.Instead,youuse ClouderaManagertoperformthesetasks. Managing CDH and Managed Services Using Cloudera Manager YoumanageCDHandmanagedservicesusingtheClouderaManagerAdminConsoleandClouderaManagerAPI. ThefollowingsectionsfocusontheClouderaManagerAdminConsole. ConfigurationOverview WhenClouderaManagerconfiguresaservice,itallocatesrolesthatarerequiredforthatservicetothehostsinyour cluster.Theroledetermineswhichservicedaemonsrunonahost. Forexample,foranHDFSserviceinstance,ClouderaManagerconfigures: • OnehosttoruntheNameNoderole. • OnehosttorunasthesecondaryNameNoderole. • OnehosttoruntheBalancerrole. • RemaininghostsastorunDataNoderoles. Arolegroupisasetofconfigurationpropertiesforaroletype,aswellasalistofroleinstancesassociatedwiththat group.ClouderaManagerautomaticallycreatesadefaultrolegroupnamedRoleTypeDefaultGroupforeachrole type. Whenyouruntheinstallationorupgradewizard,ClouderaManagerconfiguresthedefaultrolegroupsitadds,and addsanyotherrequiredrolegroupsforagivenroletype.Forexample,aDataNoderoleonthesamehostasthe NameNodemightrequireadifferentconfigurationthanDataNoderolesrunningonotherhosts.ClouderaManager createsaseparaterolegroupfortheDataNoderolerunningontheNameNodehostandusesthedefaultconfiguration forDataNoderolesrunningonotherhosts. ClouderaManagerwizardsautoconfigurerolegrouppropertiesbasedontheresourcesavailableonthehosts.For propertiesthatarenotdependentonhostresources,ClouderaManagerdefaultvaluestypicallyalignwithCDHdefault valuesforthatconfiguration.ClouderaManagerdeviateswhentheCDHdefaultisnotarecommendedconfiguration orwhenthedefaultvaluesareillegal. ClouderaManagerConfigurationLayout AfterrunningtheInstallationwizard,useClouderaManagertoreconfiguretheexistingservicesandaddandconfigure additionalhostsandservices. ClouderaManagerconfigurationscreensoffertwolayoutoptions:new(thedefault)andclassic.Youcanswitchbetween layoutsusingtheSwitchtoXXXlayoutlinkatthetoprightofthepage.Keepthefollowinginmindwhenyouselecta layout: • Ifyouswitchtotheclassiclayout,ClouderaManagerpreservesthatsettingwhenyouupgradetoanewversion. • Selectionsmadeinonelayoutarenotpreservedwhenyouswitch. • Certainfeatures,includingcontrolsforconfiguringNavigatorauditeventsandHDFSlogredaction,aresupported onlyinthenewlayout. 8|ClouderaAdministration ManagingCDHandManagedServices Newlayoutpagescontaincontrolsthatallowyoufilterconfigurationpropertiesbasedonconfigurationstatus,category, andgroup.Forexample,todisplaytheJournalNodemaximumlogsizeproperty(JournalNodeMaxLogSize),clickthe CATEGORY>JournalNodeandGROUP>Logsfilters: Whenaconfigurationpropertyhasbeensettoavaluedifferentfromthedefault,aresettodefaultvalueicon displays. Classiclayoutpagesareorganizedbyrolegroupandcategorieswithintherolegroup.Forexample,todisplaythe JournalNodemaximumlogsizeproperty(JournalNodeMaxLogSize),selectJournalNodeDefaultGroup>Logs. ClouderaAdministration|9 ManagingCDHandManagedServices Whenaconfigurationpropertyhasbeensettoavaluedifferentfromthedefault,aResettothedefaultvaluelink displays. Thereisnomechanismforresettingtoanautoconfiguredvalue.However,youcanusetheconfigurationhistoryand rollbackfeaturetorevertanyconfigurationchanges. ModifyingConfigurationPropertiesUsingClouderaManager Note: ThistopicdiscusseshowtoconfigurepropertiesusingtheClouderaManager"newlayout."Theolder layout,calledthe"classiclayout"isstillavailable.Forinstructionsonusingtheclassiclayout,see ModifyingConfigurationProperties(ClassicLayout)onpage15. Toswitchbetweenthelayouts,clickeithertheSwitchtothenewlayoutorSwitchtotheclassic layoutlinksintheupper-rightportionofallconfigurationpages. MinimumRequiredRole:Configurator(alsoprovidedbyClusterAdministrator,FullAdministrator) WhenaserviceisaddedtoClouderaManager,eitherthroughtheinstallationorupgradewizardorwiththeAdd Servicesworkflow,ClouderaManagerautomaticallysetstheconfigurationproperties,basedontheneedsoftheservice andcharacteristicsoftheclusterinwhichitwillrun.Theseconfigurationpropertiesincludebothservice-wide configurationproperties,aswellasspecificpropertiesforeachroletypeassociatedwiththeservice,managedthrough rolegroups.Arolegroupisasetofconfigurationpropertiesforaroletype,aswellasalistofroleinstancesassociated withthatgroup.ClouderaManagerautomaticallycreatesadefaultrolegroupnamedRoleTypeDefaultGroupfor eachroletype.SeeRoleGroupsonpage48. ChangingtheConfigurationofaServiceorRoleInstance 1. Gototheservicestatuspage.(Cluster>servicename) 2. ClicktheConfigurationtab. 3. Locatethepropertyyouwanttoedit.Youcantypeallorpartofthepropertynameinthesearchbox,orusethe filtersontheleftsideofthescreen: • TheStatussectionlimitsthedisplayedpropertiesbytheirstatus.Possiblestatusesinclude: 10|ClouderaAdministration