Services and State with Ambari REST API

  • 时间:
  • 浏览:0
  • 来源:大发时时彩代理—大发大发彩票app

3

5

21

Ranger Admin ServerRANGER_ADMINRANGER1-3

4

WebHcat ServerWEBHCAT_SERVERHIVE1

The possible states a service component can be in is listed below. To change or get the state of a service component the following API calls can be issued.

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/KNOX_GATEWAY

INSTALL_FAILEDInstall failed.

// Getting the state of the component

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts

Oozie ServerOOZIE_SERVEROOZIE1

/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER

UNINSTALLEDState when uninstall completed successfully.

Knox GatewayKNOX_GATEWAYKNOX1+

6

Nimbus ServerNIMBUSSTORM1

// Get service info

1

26

DRPC ServerDRPC_SERVERSTORM1

16

ZookeeperZOOKEEPER_SERVERZOOKEEPER1+ (odd #)

12

23

4

17

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/services/KNOX

DISABLEDDisabled master’s backup state.

HDFS ClientHDFS_CLIENTHDFS1+

-d '{"RequestInfo": {"context": "Install Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 

ServiceCardinality

Ranger User SyncRANGER_USERSYNCRANGER1

NameNodeNAMENODEHDFS1-2

A Hadoop cluster contains of multiple services that individually have either master, slave or client components. Below you will find a list of all currently supported components as part of a HDP stack divided in master, slave, or client groups. The cardinality notates the number of components that can exist at the same time in one cluster.

/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

INSTALLEDState when install completed successfully.

Possible states of a component:

"Body":{"HostRoles": {"state""INSTALLED"}}}' 

NameAmbari Component Name

INIT <-UNINSTALLED, INIT, WIPING_OUT

/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

STARTEDState when start completed successfully.

Mahout ClientMAHOUTMAHOUT0+

Ambari Metrics CollectorMETRICS_MONITORAMSALL

13

13

Zookeeper ClientZOOKEEPER_CLIENTZOOKEEPER1+

10

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X GET

The Ambari management tool for Hadoop offers among other handy tools a comprehensive REST API for cluster administration. Logically a cluster is divided into hosts, services and service components. While the UI might not always has support for all needed scenarios sure the REST API can be used to achieve it. For example moving a master component of a service from one host to another.

3

UNINSTALLED <-INSTALLED, UNINSTALLED, UNINSTALLING

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/

Among the need to change the state of a service component a very common issue is the need to actually move one service – most likely a master service – to different host. Users faced with this problems often need to issue the existing API due to a lack of support by the existing UI – at least with older releases of Ambari Web UI. This is an example of moving the OOZIE_SERVER to a new host.

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X PUT 

24

19

STOPPINGIn the process of stopping.

8

-d '{"RequestInfo": {"context": "Stop Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 

curl -u admin:admin -H "X-Requested-By:ambari" -i -X DELETE 

8

Ambari Metrics ServiceMETRICS_COLLECTORAMS1

Removing a complete service from the cluster would require all the removal of all it’s components together with the service itself. For example for removing the Knox service we would need to remove the KNOX_GATEWAY component from the cluster, but before let’s get their state:

-d '{"RequestInfo": {"context": "Start Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 

STARTINGIn the process of starting.

DISABLED <-INSTALLED, INSTALL_FAILED, UNKNOWN

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts

Phoneix Query ServerPHOENIX_QUERY_SERVERHBASE0+

// Starting the new component

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X DELETE

Kafka BrokerKAFKA_BROKERKAFKA1+

Sqoop ClientSQOOPSQOOP1+

UNINSTALLINGIn the process of uninstalling.

/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

-d '{"ServiceInfo": {"state": "INSTALLED"}}'

Falcon ServerFALCON_SERVERFALCON1

curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

17

9

"Body":{"HostRoles": {"state""STARTED"}}}' 

HBase RegionServerHBASE_REGIONSERVERHBASE1+

StateDescription

STARTED <-INSTALLED, STARTING, STARTED

HiveServer2HIVE_SERVERHIVE1-2

20

7

HBase MasterHBASE_MASTERHBASE1+

14

In this post we are going to look a little closer at the way the Ambari API can be used to manage Hadoop services. At the end of this post you will find a list of all the currently supported Hadoop services with all the needed master, slave and client components that can be manged and administrated within your HDP stack. Also this posts contains the possible states and state transitions a component might have which could become useful when facing problems like Host config is in invalid state.

INSTALLED <-INIT, UNINSTALLED, INSTALLED, INSTALLING, STARTED, INSTALL_FAILED, UPGRADING, STOPPING, UNKNOWN, DISABLED

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X PUT 

25

Hive ClientHIVE_CLIENTHIVE1+

HBase ClientHBASE_CLIENTHBASE1+

2

curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

Desired StateStart State

ServiceCardinality

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/services/KNOX

5

DataNodeDATANODEHDFS1+

Secondary NameNodeNFS_GATEWAYHDFS0+

2

MapReduce ClientMAPREDUCE2_CLIENTMAPREDUCE21+

Storm SupervisorSUPERVISORSTORM1+

// Add component to host

-d '{"RequestInfo":{"context":"Install YARN_CLIENT"},

10

9

"Body":{"HostRoles":{"state":"INSTALLED"}}}' 

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts

5

Ranger Key Management ServerRANGER_KMS_SERVERRANGER_KMS1+

8

ResourceMangerRESOURCEMANAGERYARN1-2

curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

14

Storm REST ServerSTORM_REST_APISTORM1

Zeppelin ServerZEPPELIN_MASTERSPARK / HIVE1

curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

"Body":{"HostRoles": {"state""INSTALLED"}}}' 

Storm UISTORM_UI_SERVERSTORM1

HistoryServerHISTORYSERVERMAPREDUCE2 1

Node ManagerNODEMANAGERYARN1+

Application Timeline ServerAPP_TIMELINE_SERVERYARN1

18

Ganglia ServerGANGLIA_SERVERGANGLIA1

6

11

// Delete component

1

2

Falcon ClientFALCON_CLIENTFALCON1+

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/components/KNOX_GATEWAY

// Install component by achieving stated INSTALLED

curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d 

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X DELETE

Zookeeper Failover ServiceZKFCHDFS0+

22

YARN ClientYARN_CLIENTYARN1+

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts

/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER

15

16

UPGRADINGIn the process of upgrading the deployed bits.

7

// Stop component KNOX_GATEWAY

HCat ClientHCATHIVE1+

Nagios ServerNAGIOS_SERVERNAGIOS1

15

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT

12

NameAmbari Component Name

1

UNKNOWNState could not be determined.

Cardinality

// Delete service

// Delete old component

6

curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X GET

Spark ClientSPARK_CLIENTSPARK1+

Hive MetastoreHIVE_METASTOREHIVE1-2

4

WIPING_OUTIn the process of wiping out the install.

NameAmbari Component Name

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts

Ganglia Metrics CollectorGANGLIA_MONITORGANGLIAALL

// Setting the state of the component to INSTALLED

// Stop service KNOX

Journale Nodes for NameNode HAJOURNALNODEHDFS0+ (odd #)

3

11

-d '{"ServiceComponentInfo": {"state": "INSTALLED"}}'

Allowed State transitions of a component:

INITInitial/Clean state.

http://<HOST>:400400/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT 

Service

7

INSTALLINGIn the process of installing.

// Stop the old component by putting it back in state INSTALLED

Secondary NameNodeSECONDARY_NAMENODEHDFS1

Oozie ClientOOZIE_CLIENTOOZIE1+

Spark Job History ServerSPARK_JOBHISTORYSERVERSPARK1