본문 바로가기

Spring Cloud

Spring Cloud Service Discovery - Netflix Eureka (1)

전체 내용은 Github에 있습니다 :)


목차

  1. DiscoveryClient
  2. Netflix eureka 개요
  3. Netflix eureka 시작하기

DiscoveryClient

서비스 레지스트리는 서비스 인스턴스와 서비스가 제공하는 API를 포함하는 테이블입니다.
(서비스 레지스트리는 CAP(Consistency 일관성, Availability 가용성, Partition tolerance 분리 내구성) 정리의 제약을 받는다고 합니다. CAP 이론을 정확히 알지못하지만 3가지를 만족하는 분산 시스템은 존재하지 않는다? 그래서 상황에 따라 2개를 만족하는 시스템을 선택한다? 이런 내용인 것 같습니다)

Spring Cloud는 DiscoveryClient 추상화를 통해 다양한 유형의 서비스 레지스트리를 이용할 수 있습니다. (Cloud foundry, Zookeper, Consul, Eureka, ETCD 등)

DiscoveryClient

package org.springframework.cloud.client.discovery;

import java.util.List;

import org.springframework.cloud.client.ServiceInstance;
import org.springframework.core.Ordered;

public interface DiscoveryClient extends Ordered {

    /**
     * Default order of the discovery client.
     */
    int DEFAULT_ORDER = 0;

    /**
     * A human-readable description of the implementation, used in HealthIndicator.
     * @return The description.
     */
    String description();

    /**
     * Gets all ServiceInstances associated with a particular serviceId.
     * @param serviceId The serviceId to query.
     * @return A List of ServiceInstance.
     */
    List<ServiceInstance> getInstances(String serviceId);

    /**
     * @return All known service IDs.
     */
    List<String> getServices();

    /**
     * Default implementation for getting order of discovery clients.
     * @return order
     */
    @Override
    default int getOrder() {
        return DEFAULT_ORDER;
    }
}

위의 클래스를 살펴보면, getInstances(serviceId) 메소드를 통해서 특정 서비스ID를 이용하는 모든 서버 정보를, getServices()를 통해 모든 서비스 ID값을 조회할 수 있습니다.

다음으로 살펴볼 클래스는 Spring cloud에서 정의한 ServiceInstance입니다.
아래와 같이 instance id, service id, host, port 등의 정보를 제공하도록 정의하였습니다.

/**
 * Represents an instance of a service in a discovery system.
 */
public interface ServiceInstance {

    /**
     * @return The unique instance ID as registered.
     */
    default String getInstanceId() {
        return null;
    }

    /**
     * @return The service ID as registered.
     */
    String getServiceId();

    /**
     * @return The hostname of the registered service instance.
     */
    String getHost();

    /**
     * @return The port of the registered service instance.
     */
    int getPort();

    /**
     * @return Whether the port of the registered service instance uses HTTPS.
     */
    boolean isSecure();

    /**
     * @return The service URI address.
     */
    URI getUri();

    /**
     * @return The key / value pair metadata associated with the service instance.
     */
    Map<String, String> getMetadata();

    /**
     * @return The scheme of the service instance.
     */
    default String getScheme() {
        return null;
    }
}

마지막으로 DiscoveryClient 구현체인 EurekaDiscoveryClient를 살펴보면,
com.netflix.discovery.EurekaClient를 이용하여 InstanceInfo 정보를 가져와 ServiceInstance로 컨버팅 해주고 있습니다.

public class EurekaDiscoveryClient implements DiscoveryClient {
    ...
    @Override
    public List<ServiceInstance> getInstances(String serviceId) {
        List<InstanceInfo> infos = this.eurekaClient.getInstancesByVipAddress(serviceId,
                false);
        List<ServiceInstance> instances = new ArrayList<>();
        for (InstanceInfo info : infos) {
            instances.add(new EurekaServiceInstance(info));
        }
        return instances;
    }

    @Override
    public List<String> getServices() {
        Applications applications = this.eurekaClient.getApplications();
        if (applications == null) {
            return Collections.emptyList();
        }
        List<Application> registered = applications.getRegisteredApplications();
        List<String> names = new ArrayList<>();
        for (Application app : registered) {
            if (app.getInstances().isEmpty()) {
                continue;
            }
            names.add(app.getName().toLowerCase());

        }
        return names;
    }
    ...
}

Netflix Eureka 개요

01_eureka_highlevel_archi

위의 그림을 살펴보면 아래와 같은 서비스들이 상호작용하는 것을 볼 수 있습니다.

  1. Eureka Server : 서비스 레지스트리를 관리해주는 서버
  2. Application Service : Eureka Server에 자신을 등록한 뒤 Service Discovery의 대상인 서버
  3. Application Client : Eureka Server에서 Service Discovery 후 API 요청

위의 그림에서 살펴본 Resiger, Renew, Fetch Registry, Cancel에 대해서 살펴보겠습니다.
(자세한 내용은 Eureka wiki-Understanding-eureka-client-server-communication에 있습니다.)

1. Register

Client가 Server에게 등록 요청을 보내서 서버의 서비스 레지스트리에 등록하는 행위 입니다.

2. Renew

30초마다 Client는 Server에게 Heartbeat를 보내 lease를 갱신합니다. Server는 마지막 Heartbeat를 보낸시간보다 90초가 지났으면 해당 서비스를 Registry에서 제거합니다.

3. Fetch Registry

Client는 Server로부터 Registry 정보를 가져와서 로컬 캐시에 담아둡니다. 이러한 Registry 정보는 service discovery하는데 사용됩니다. 아래의 com.netflix.discovery.DiscoveryClient를 살펴보면 delta를 조회하여 client의 이전 delta와 해시코드(문자열)를 비교하여 변경사항이 있으면 갱신하는 코드를 볼 수 있습니다.

private void getAndUpdateDelta(Applications applications) throws Throwable {
    long currentUpdateGeneration = fetchRegistryGeneration.get();

    Applications delta = null;
    // /apps/delta?regions=... 호출
    EurekaHttpResponse<Applications> httpResponse = eurekaTransport.queryClient.getDelta(remoteRegionsRef.get());
    if (httpResponse.getStatusCode() == Status.OK.getStatusCode()) {
        delta = httpResponse.getEntity();
    }

    if (delta == null) {
        logger.warn("The server does not allow the delta revision to be applied because it is not safe. "
                + "Hence got the full registry.");
        getAndStoreFullRegistry();
    } else if (fetchRegistryGeneration.compareAndSet(currentUpdateGeneration, currentUpdateGeneration + 1)) {
        logger.debug("Got delta update with apps hashcode {}", delta.getAppsHashCode());
        String reconcileHashCode = "";
        if (fetchRegistryUpdateLock.tryLock()) {
            try {
                // client의 delta 갱신
                updateDelta(delta);
                // {instance_name}:{count}_{instance_name}_{count} ... 
                reconcileHashCode = getReconcileHashCode(applications);
            } finally {
                fetchRegistryUpdateLock.unlock();
            }
        } else {
            logger.warn("Cannot acquire update lock, aborting getAndUpdateDelta");
        }
        ...
    } 
    ...
}

4. Cancel

Client는 Server에게 Cancel 요청을 보내 서버의 Registry에서 제거할 수 있습니다. (주로 애플리케이션 종료 시)


 

Client와 Server의 상호작용을 정리하면 아래와 같습니다.

Eureka interacts

02_eureka_interact

  1. Service1 : 애플리케이션 시작 시 Eureka 서버에서 Register 요청을 보낸 뒤 서버는 Registry에 추가한다.
  2. Service2 : 주기적으로 Heatbeat를 보낸다.(Renew)
  3. Service3 : 애플리케이션 종료 시 Eureka 서버에게 Cancel 요청을 보낸 뒤 서버는 Registry에서 제거한다.
  4. Service4 : 애플리에키션 종료 시 Cancel 요청을 보내지 않은 상태이다.
  5. Eureka Server : Service4에 대하여 마지막 Heatbeat로 부터 90초가 지나서 Registry에서 제거한다.

더 자세한 Endpoint는 WIKI-Eureka-REST-operations 를 확인하시면 됩니다 :)


Netflix eureka 시작하기

간단하게 Standalone mode의 Eureka Server를 실행하고 Eureka client를 포함하고 있는 Account-Service를 실행해보겠습니다.

Eureka Server 설정하기(간단!)

application.yaml

eureka:
  # dashboard에 대한 설정으로, http://localhost:3000/eureka-ui 를 통해 확인할 수 있다.
  dashboard:
    path: /eureka-ui
  instance:
    hostname: localhost
    statusPageUrlPath: /info
    healthCheckUrlPath: /health
  # 등록된 인스턴스 중 많은 수가 정해진 시간 내에 Heatbeat를 보내지 않으면 Eureka는 이를 인스턴스 문제가 아닌
  # 네트워크 문제라고 간주하고 Registry를 그대로 유지한다. Example 실행을 위해 false로 설정
  server:
    enableSelfPreservation: false
  client:
    # Eureka client -> Eureka server로 등록 여부
    # standalone mode이므로 자기 자신을 등록할 필요가 없다.
    registerWithEureka: false
    # Eureka Client -> Eureka server로 Registry fetch 여부
    fetchRegistry: false

@EnableEurekaServer

package server;

import org.springframework.boot.WebApplicationType;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import lombok.extern.slf4j.Slf4j;

@Slf4j
@SpringBootApplication
@EnableEurekaServer
public class EurekaServerApplication {
    public static void main(String[] args) {
        new SpringApplicationBuilder(EurekaServerApplication.class)
                .web(WebApplicationType.SERVLET).run(args);
    }
}

Eureka Client 설정하기

bootstrap.yaml

spring:
  application:
    name: account-service
eureka:
  instance:
    # 랜덤값을 이용하여 instance id를 고유하게 재정의
    instance-id: ${spring.application.name}:${spring.application.instance_id:${random.value}}

application.yaml

eureka:
  # service instance에 대한 설정
  instance:
    statusPageUrlPath: /actuator/info
    healthCheckUrlPath: /actuator/health
  # eureka client 설정
  client:
    serviceUrl:
      defaultZone: http://localhost:3000/eureka/
# /actuator/info 호출 시 출력 된 Application 정보
info:
  app:
    name: Account Example Application
    version: 1.0.0
    discription: This is a demo project for eurkea

실행하기

해당 Github에 간단하게 start.sh를 작성해놨는데요, 아래와 같이 실행할 수 있습니다.

start server/client

// eureka 서버 시작하기
$ ./tools/script/start.sh server  

// account 서버1 시작하기(profile:default, port:3100)
$ ./tools/script/start.sh account default 3100

// account 서버2 시작하기(profile:default, port:3101)
$ ./tools/script/start.sh server default 3101

확인하기

Eureka server dashboard 확인하기

http://localhost:3000/eureka-ui를 접속하면 아래와 같은 화면을 확인할 수 있습니다.
(server application.yaml의 eureka.dashboard.path 값)

03_eureka_dashboard

DiscoveryClient의 ServiceInstance 리스트 조회하기

Eureka Client를 가지고 있는 Account-Service에서 아래와 같은 코드를 추가해 /discovery/services를 호출하면 모든 인스턴스를 반환하는 API를 호출하였습니다.

```java @Autowired private DiscoveryClient discoveryClient;

@GetMapping("/discovery/services")
public Map<String, List> discoveryServices() {
return discoveryClient.getServices()
.stream()
.collect(Collectors.toMap(s -> s, s -> discoveryClient.getInstances(s)));
}


아래와 같이 요청하면 Spring cloud의 `ServiceInstance` 구현체인 `EurekaServiceInstance`의 응답 결과를 확인할 수 있습니다.
(instanceId ~ metadata는 ServiceInstance 스펙이고 instanceInfo는 eureka의 InstanceInfo 스펙이다.)  

```bash
$ curl -XGET http://localhost:3100/discovery/services | jq . 
{
  "account-service": [
    {
      "instanceId": "account-service:b12e61ad1432e7da69d396f443e8b1bf",
      "serviceId": "ACCOUNT-SERVICE",
      "host": "192.168.0.2",
      "port": 3101,
      "secure": false,
      "scheme": "http",
      "uri": "http://192.168.0.2:3101",
      "metadata": {
        "management.port": "3101"
      },
      "instanceInfo": {
        "instanceId": "account-service:b12e61ad1432e7da69d396f443e8b1bf",
        "app": "ACCOUNT-SERVICE",
        "appGroupName": null,
        "ipAddr": "192.168.0.2",
        "sid": "na",
        "homePageUrl": "http://192.168.0.2:3101/",
        "statusPageUrl": "http://192.168.0.2:3101/actuator/info",
        "healthCheckUrl": "http://192.168.0.2:3101/actuator/health",
        "secureHealthCheckUrl": null,
        "vipAddress": "account-service",
        "secureVipAddress": "account-service",
        "countryId": 1,
        "dataCenterInfo": {
          "@class": "com.netflix.appinfo.InstanceInfo$DefaultDataCenterInfo",
          "name": "MyOwn"
        },
        "hostName": "192.168.0.2",
        "status": "UP",
        "overriddenStatus": "UNKNOWN",
        "leaseInfo": {
          "renewalIntervalInSecs": 30,
          "durationInSecs": 90,
          "registrationTimestamp": 1598537803756,
          "lastRenewalTimestamp": 1598537923714,
          "evictionTimestamp": 0,
          "serviceUpTimestamp": 1598537803756
        },
        "isCoordinatingDiscoveryServer": false,
        "metadata": {
          "management.port": "3101"
        },
        "lastUpdatedTimestamp": 1598537803756,
        "lastDirtyTimestamp": 1598537803712,
        "actionType": "ADDED",
        "asgName": null
      }
    },
    {...}
  ]
}

Server/Client 로그 확인하기

  • (1) Client는 GET /eureka//apps/?를 호출하여 Registry 정보를 가져오는 것을 확인할 수 있습니다.
// Client log
2020-08-27 23:36:57.663  INFO 17140 --- [           main] com.netflix.discovery.DiscoveryClient    : Getting all instance registry info from the eureka server
...
2020-08-27 23:36:57.841 DEBUG 17140 --- [           main] n.d.s.t.j.AbstractJerseyEurekaHttpClient : Jersey HTTP GET http://localhost:3000/eureka//apps/?; statusCode=200
2020-08-27 23:36:57.841 DEBUG 17140 --- [           main] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Pinning to endpoint null
2020-08-27 23:36:57.841  INFO 17140 --- [           main] com.netflix.discovery.DiscoveryClient    : The response status is 200
2020-08-27 23:36:57.841 DEBUG 17140 --- [           main] com.netflix.discovery.DiscoveryClient    : Got full registry with apps hashcode 
2020-08-27 23:36:57.842 DEBUG 17140 --- [           main] com.netflix.discovery.DiscoveryClient    : The total number of all instances in the client now is 0


// Server log
2020-08-27 23:36:57.797 DEBUG 17119 --- [nio-3000-exec-1] c.n.e.registry.AbstractInstanceRegistry  : Fetching applications registry with remote regions: false, Regions argument []
2020-08-27 23:36:57.803 DEBUG 17119 --- [nio-3000-exec-1] c.n.eureka.registry.ResponseCacheImpl    : New application cache entry {name=ALL_APPS, type=Application, format=JSON} with apps hashcode
2020-08-27 23:36:57.940 DEBUG 17119 --- [nio-3000-exec-2] c.n.d.util.DeserializerStringCache       : clearing global-level cache with size 1
2020-08-27 23:36:57.940 DEBUG 17119 --- [nio-3000-exec-2] c.n.d.util.DeserializerStringCache       : clearing app-level serialization cache with size 8
  • (2) Client는 POST /eureka//apps/ACCOUNT-SERVICE를 호출하여 자신의 인스턴스를 Register 후 204 Status code로 성공응답을 받는 것을 확인할 수 있습니다.
// client log
2020-08-27 23:36:57.843  INFO 17140 --- [           main] com.netflix.discovery.DiscoveryClient    : Starting heartbeat executor: renew interval is: 30
...
2020-08-27 23:36:57.852  INFO 17140 --- [           main] o.s.c.n.e.s.EurekaServiceRegistry        : Registering application ACCOUNT-SERVICE with eureka with status UP
...
2020-08-27 23:36:57.855  INFO 17140 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff: registering service...
...
2020-08-27 23:36:57.954 DEBUG 17140 --- [nfoReplicator-0] n.d.s.t.j.AbstractJerseyEurekaHttpClient : Jersey HTTP POST http://localhost:3000/eureka//apps/ACCOUNT-SERVICE with instance account-service:a389e2d6e5bf85f9e1544048fbbf8eff; statusCode=204
2020-08-27 23:36:57.954 DEBUG 17140 --- [nfoReplicator-0] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Pinning to endpoint null
2020-08-27 23:36:57.954  INFO 17140 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff - registration status: 204

// server log
2020-08-27 23:36:57.940 DEBUG 17119 --- [nio-3000-exec-2] c.n.e.resources.ApplicationResource      : Registering instance account-service:a389e2d6e5bf85f9e1544048fbbf8eff (replication=null)
2020-08-27 23:36:57.940 DEBUG 17119 --- [nio-3000-exec-2] o.s.c.n.eureka.server.InstanceRegistry   : register ACCOUNT-SERVICE, vip account-service, leaseDuration 90, isReplication false
2020-08-27 23:36:57.941 DEBUG 17119 --- [nio-3000-exec-2] c.n.e.registry.AbstractInstanceRegistry  : No previous lease information found; it is new registration
2020-08-27 23:36:57.941 DEBUG 17119 --- [nio-3000-exec-2] c.n.e.registry.AbstractInstanceRegistry  : Processing override status using rule: [com.netflix.eureka.registry.rule.DownOrStartingRule, com.netflix.eureka.registry.rule.OverrideExistsRule, com.netflix.eureka.registry.rule.LeaseExistsRule, com.netflix.eureka.registry.rule.AlwaysMatchInstanceStatusRule]
2020-08-27 23:36:57.941 DEBUG 17119 --- [nio-3000-exec-2] c.n.e.r.r.AlwaysMatchInstanceStatusRule  : Returning the default instance status UP for instance account-service:a389e2d6e5bf85f9e1544048fbbf8eff
2020-08-27 23:36:57.942 DEBUG 17119 --- [nio-3000-exec-2] c.n.eureka.registry.ResponseCacheImpl    : Invalidating the response cache key : Application ACCOUNT-SERVICE V1 JSON, full
2020-08-27 23:36:57.943 DEBUG 17119 --- [nio-3000-exec-2] c.n.eureka.registry.ResponseCacheImpl    : Invalidating the response cache key : Application ACCOUNT-SERVICE V1 JSON, compact
...
  • (3) Client는 PUT /eureka//apps/ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff를 호출하여 Heartbeat를 보내 Renew 작업이 이루어지는것을 확인할 수 있습니다.
// client log
2020-08-27 23:37:27.878 DEBUG 17140 --- [tbeatExecutor-0] n.d.s.t.j.AbstractJerseyEurekaHttpClient : Jersey HTTP PUT http://localhost:3000/eureka//apps/ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff; statusCode=200
2020-08-27 23:37:27.879 DEBUG 17140 --- [tbeatExecutor-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff - Heartbeat status: 200
2020-08-27 23:37:27.897 DEBUG 17140 --- [freshExecutor-0] c.n.d.shared.MonitoredConnectionManager  : Released connection is reusable.
...
2020-08-27 23:37:27.897 DEBUG 17140 --- [freshExecutor-0] n.d.s.t.j.AbstractJerseyEurekaHttpClient : Jersey HTTP GET http://localhost:3000/eureka//apps/?; statusCode=200
2020-08-27 23:37:27.897  INFO 17140 --- [freshExecutor-0] com.netflix.discovery.DiscoveryClient    : The response status is 200

// server log
2020-08-27 23:37:27.876 DEBUG 17119 --- [nio-3000-exec-4] o.s.c.n.eureka.server.InstanceRegistry   : renew ACCOUNT-SERVICE serverId account-service:a389e2d6e5bf85f9e1544048fbbf8eff, isReplication {}false
2020-08-27 23:37:27.876 DEBUG 17119 --- [nio-3000-exec-4] c.n.e.registry.AbstractInstanceRegistry  : Fetching applications registry with remote regions: false, Regions argument []
  • (4) Client는 DELETE /eureka//apps/ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff를 호출하여 Cancel 요청을 보낸 후 서버는 Registry에서 제거 되는것을 확인할 수 있습니다.
// client log
2020-08-28 00:02:33.149  INFO 19325 --- [extShutdownHook] o.s.c.n.e.s.EurekaServiceRegistry        : Unregistering application ACCOUNT-SERVICE with eureka with status DOWN
...
2020-08-28 00:02:33.149  INFO 19325 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff: registering service...
...
2020-08-28 00:02:36.187 DEBUG 19325 --- [extShutdownHook] n.d.s.t.j.AbstractJerseyEurekaHttpClient : Jersey HTTP DELETE http://localhost:3000/eureka//apps/ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff; statusCode=200
2020-08-28 00:02:36.187  INFO 19325 --- [extShutdownHook] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff - deregister  status: 200

// server log
2020-08-28 00:02:36.184 DEBUG 19308 --- [nio-3000-exec-6] o.s.c.n.eureka.server.InstanceRegistry   : cancel ACCOUNT-SERVICE, serverId account-service:a389e2d6e5bf85f9e1544048fbbf8eff, isReplication false
2020-08-28 00:02:36.185 DEBUG 19308 --- [nio-3000-exec-6] c.n.eureka.registry.ResponseCacheImpl    : Invalidating the response cache key : Application ACCOUNT-SERVICE V1 JSON, full
...
2020-08-28 00:02:36.186  INFO 19308 --- [nio-3000-exec-6] c.n.e.registry.AbstractInstanceRegistry  : Cancelled instance ACCOUNT-SERVICE/account-service:a389e2d6e5bf85f9e1544048fbbf8eff (replication=false)
2020-08-28 00:02:36.186 DEBUG 19308 --- [nio-3000-exec-6] c.n.eureka.resources.InstanceResource    : Found (Cancel): ACCOUNT-SERVICE - account-service:a389e2d6e5bf85f9e1544048fbbf8eff

다음 글에서는 Eureka Server의 HA 구성과 어떻게 서버간 통신하는지 작성하겠습니다 :)

'Spring Cloud' 카테고리의 다른 글

Spring Cloud Service Discovery - Netflix Eureka (2)  (0) 2020.12.27