Amplify Central Save PDF Selected topic Selected topic and subtopics All content Service Mesh Traceability Observe transactions in mesh. 7 minute read Before you begin Before you start, see Deploy your agents with the Axway CLI to learn how to use the CLI to install the Istio agents into your Kubernetes cluster. This page will reference the resources created from the Deploy your agents with the Axway CLI procedure. Prerequisites These prerequisites are required by the Axway Central CLI, which you will use to configure the Amplify Istio Discovery Agent. Node.js >= 10.13.0 and <= 12.14.1 Minimum Axway Central CLI version: 1.7.0 or later For more information, see Install Axway Central CLI. Overview The Amplify Istio Traceability Agent is installed into your Kubernetes cluster as part of deploying the ampc-hybrid helm chart. The Traceability Agent (TA) sends metrics and logs for API activity back to Amplify Central so that you can monitor service activity and troubleshoot your services. The agent publishes a summary of the transaction which can be seen in Business Insights. Once the transaction summary is expanded, you can see all the related spans within a transaction including the request and response headers for each. The Amplify Istio Traceability Agent has two modes; default and verbose. The default mode captures only the headers specified in the EnvoyFilter and the verbose mode captures all the headers in request and response flows. Setup The Amplify Istio Traceability Agent logs and publishes traffic within the Mesh. In order to generate traffic, we need to create certain custom resource definitions (CRDs) in the mesh. Amplify Central resources In order to better filter transactions related to the services in the mesh, certain resources need to be created for each service running on the mesh - namely APIService, APIServiceRevision and APIServiceInstance. The APIService needs to include the attribute “externalAPIID” in its definition. The format of externalAPIID is: clustername-httproutename The “externalAPIID” attribute is used to define the correlation between the Kubernetes cluster name and the http route name on the VirtualService (that is created in the next section) so that the traffic for this APIService can be easily filtered out. The cluster name is the value of the field als.clusterName in the agent override file hybrid-override.yaml generated by the Axway CLI during Istio agent installation to your Kubernetes cluster. Choose a name for the http route name - ideally the same as the name for APIService or very similar to it. Make a note of this route name, as you will need it in the next section. Next, create the following resources in Central using the Axway CLI: Note Create these resources manually only if you are running standalone Amplify Istio Traceability Agent. If you are running Amplify Istio Traceability Agent along with the Discovery agents, these resources are automatically created and you can skip to Istio CRDs. kind: APIService name: <<serviceName>> metadata: scope: kind: Environment name: <<environmentName>> attributes: externalAPIID: <<clustername-http.name>> # http.name to be used for VS spec: {} --- kind: APIServiceRevision name: <<revisionName>> metadata: scope: kind: Environment name: <<environmentName>> spec: apiService: <<serviceName>> definition: type: "oas2" value: <<base64 encoded spec>> --- kind: APIServiceInstance name: <<serviceName>> metadata: scope: kind: Environment name: <<environmentName>> spec: apiServiceRevision: <<revisionName>> endpoint: - host: "apicentral.axway.com" port: 8080 protocol: http routing: basePath: <<basePath>> Here is an example for the Hybrid List demo service: kind: APIService name: mylist metadata: scope: kind: Environment name: meshone attributes: externalAPIID: clusterone-mylist spec: {} --- kind: APIServiceRevision name: list-v1 metadata: scope: kind: Environment name: meshone spec: apiService: mylist definition: type: "oas2" value: eyJzd2FnZ2VyIjoiMi4wIiwiaW5mbyI6eyJ0aXRsZSI6Im15bGlzdCIsImRlc2NyaXB0aW9uIjoiQW4gQVBJIEJ1aWxkZXIgc2VydmljZSIsInZlcnNpb24iOiIxLjAuMCJ9LCJob3N0IjoiMTAwLjk5LjIyMi4xMDI6ODA4MCIsImJhc2VQYXRoIjoiL2FwaSIsInBhdGhzIjp7Ii9saXN0Ijp7ImdldCI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsidHlwZSI6ImFycmF5IiwiaXRlbXMiOnsiJHJlZiI6IiMvZGVmaW5pdGlvbnMvbGlzdCJ9fX19fSwicG9zdCI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsidHlwZSI6Im9iamVjdCIsInByb3BlcnRpZXMiOnt9fX19LCJwYXJhbWV0ZXJzIjpbeyJpbiI6ImJvZHkiLCJuYW1lIjoiYm9keSIsInNjaGVtYSI6eyIkcmVmIjoiIy9kZWZpbml0aW9ucy9saXN0In19XX19LCIvbGlzdC97aWR9Ijp7ImdldCI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsiJHJlZiI6IiMvZGVmaW5pdGlvbnMvbGlzdCJ9fX0sInBhcmFtZXRlcnMiOlt7ImluIjoicGF0aCIsIm5hbWUiOiJpZCIsInR5cGUiOiJzdHJpbmciLCJyZXF1aXJlZCI6dHJ1ZX1dfSwicGFyYW1ldGVycyI6W3sibmFtZSI6ImlkIiwiaW4iOiJwYXRoIiwidHlwZSI6InN0cmluZyIsInJlcXVpcmVkIjp0cnVlfV0sImRlbGV0ZSI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsidHlwZSI6Im9iamVjdCIsInByb3BlcnRpZXMiOnt9fX19LCJwYXJhbWV0ZXJzIjpbeyJpbiI6InBhdGgiLCJuYW1lIjoiaWQiLCJ0eXBlIjoic3RyaW5nIiwicmVxdWlyZWQiOnRydWV9XX19fSwiZGVmaW5pdGlvbnMiOnsibGlzdCI6eyJ0eXBlIjoib2JqZWN0IiwidGl0bGUiOiJsaXN0IiwicHJvcGVydGllcyI6eyJuYW1lIjp7InR5cGUiOiJzdHJpbmcifSwicHJpY2UiOnsidHlwZSI6InN0cmluZyJ9LCJzdG9yZSI6eyJ0eXBlIjoic3RyaW5nIn19fSwiUmVzcG9uc2VNb2RlbCI6eyJ0eXBlIjoib2JqZWN0IiwicmVxdWlyZWQiOlsic3VjY2VzcyIsInJlcXVlc3QtaWQiXSwiYWRkaXRpb25hbFByb3BlcnRpZXMiOmZhbHNlLCJwcm9wZXJ0aWVzIjp7ImNvZGUiOnsidHlwZSI6ImludGVnZXIiLCJmb3JtYXQiOiJpbnQzMiJ9LCJzdWNjZXNzIjp7InR5cGUiOiJib29sZWFuIiwiZGVmYXVsdCI6ZmFsc2V9LCJyZXF1ZXN0LWlkIjp7InR5cGUiOiJzdHJpbmcifSwibWVzc2FnZSI6eyJ0eXBlIjoic3RyaW5nIn0sInVybCI6eyJ0eXBlIjoic3RyaW5nIn19fSwiRXJyb3JNb2RlbCI6eyJ0eXBlIjoib2JqZWN0IiwicmVxdWlyZWQiOlsibWVzc2FnZSIsImNvZGUiLCJzdWNjZXNzIiwicmVxdWVzdC1pZCJdLCJwcm9wZXJ0aWVzIjp7ImNvZGUiOnsidHlwZSI6ImludGVnZXIiLCJmb3JtYXQiOiJpbnQzMiJ9LCJzdWNjZXNzIjp7InR5cGUiOiJib29sZWFuIiwiZGVmYXVsdCI6ZmFsc2V9LCJyZXF1ZXN0LWlkIjp7InR5cGUiOiJzdHJpbmcifX19fX0= --- kind: APIServiceInstance name: mylist metadata: scope: kind: Environment name: mesh spec: apiServiceRevision: list-v1 endpoint: - host: "apicentral.axway.com" port: 8080 protocol: http routing: basePath: "/mylist" Once configured, use the following command to populate the resources in Amplify Central: axway central apply -f <fileName>.yaml Istio CRDs Gateway First, create a Gateway in the namespace in which the Istio agents were installed. If you already have a Gateway CRD, you can skip to Virtual Service and specify that Gateway in the Virtual Service. In the example below, the selector is specified as the “istio-apic-ingress” (the Ingress gateway that is installed during the Istio installation step in Deploy your agents with the Axway CLI). If you have a separate Ingress gateway that you would like to use, change the spec.selector.istio field to the label of that Ingress gateway instead. Note For more information about Gateway CRD, please refer to Istio documentation. apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: gateway-ingress namespace: <<apic-control>> spec: selector: istio: istio-apic-ingress servers: - hosts: - <<cluster-name>>.hybrid.sandbox.axwaytest.net port: name: <<port-HTTP>> number: <<8080>> protocol: <<HTTP>> For an HTTPS Gateway, use the following configuration: apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: gateway-ingress namespace: <<apic-control>> spec: selector: istio: istio-apic-ingress servers: - hosts: - <<cluster-name>>.hybrid.sandbox.axwaytest.net port: name: port-HTTP number: 443 protocol: HTTPS tls: mode: SIMPLE serverCertificate: /etc/istio/istio-ingressgateway/tls.crt privateKey: /etc/istio/istio-ingressgateway/tls.key Once configured, create the resource using the command: kubectl apply -f <fileName>.yaml Virtual Service Next, create the Virtual Service for the included demo service within the mesh. Unless you have already taken note of the ‘http route name’ part of the externalAPIId attribute associated with the APIService from the previous section, the value for it needs to be extracted from the APIService. First, locate the APIService and make a note of its name and the name of the environment under which it is scoped: axway central get apisvc Then, using this command, print the value for externalAPIID and make a note of the ‘http route name’ part of it: axway central get apisvc <APIService_name> -s <Environment_name> -o yaml | grep externalAPIID Example: axway central get apisvc ✔ Resource(s) successfully retrieved NAME AGE TITLE RESOURCE KIND SCOPE KIND SCOPE NAME RESOURCE GROUP mylist 2 days ago mylist APIService Environment cli-1618415896316 managementaxway central get apisvc mylist -s cli-1618415896316 -o yaml | grep externalAPIID externalAPIID: mycluster-mylist The following example, used to create a VirtualService, applies to the “list” demo service that comes with the Istio agent Helm installation. If you already have a Virtual Service, you can skip to the Pre-existing Virtual Services section below: apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: route-to-list namespace: apic-control spec: hosts: - "*" gateways: - gateway-ingress http: - name: mylist # this should be gotten from the http route name part of the externalAPIId on APIService match: - uri: prefix: /mylist rewrite: uri: /api route: - destination: host: ampc-hybrid-list.apic-demo.svc.cluster.local port: number: 8080 Once configured, create the resource using the command: kubectl apply -f <fileName>.yaml Pre-existing Virtual Service If you have a Virtual Service resource already, simply add a name for (or rename) the http route so that the API Service and the related transactions can be linked in API Central: Example: http: - name: mylist Note The name specified under the http.name field of the VirtualService should be the same as the ‘http route name’ part of the externalAPIId attribute on the APIService. Service Entry If you have an egress hop from a service in the mesh, then create a service entry. See the example below: apiVersion: networking.istio.io/v1beta1 kind: ServiceEntry metadata: name: httpbin.org namespace: apic-control spec: hosts: - httpbin.org ports: - name: http-80 number: 80 protocol: HTTP resolution: DNS The setup is complete for observability in the mesh. To view transactions in Business Insights, generate some traffic for the Hybrid List demo service: curl -v http://demo.sandbox.axwaytest.net:8080/mylist/list Toggling the Traceability Agent After deploying the ampc-hybrid helm chart to your Kubernetes cluster, you can see the Amplify Istio Traceability Agent running. The service is called ampc-hybrid-als. During the step Deploy your agents with the Axway CLI, you were able to select the mode for the Amplify Istio Traceability Agent. If you want to switch the mode, use the following procedure. From default to verbose: Edit the istio-override.yaml file’s configuration under the meshConfig section to set enableEnvoyAccessLogService as true: spec: meshConfig: enableTracing: true enableEnvoyAccessLogService: true After the change, re-install Istio again: istioctl install --set profile=demo -f istio-override.yaml After the Istio re-installation, run the following command to set the Amplify Istio Traceability Agent’s mode to “verbose”: helm repo update helm upgrade --install --namespace apic-control ampchybrid axway/ampc-hybrid -f hybrid-override.yaml --set als.mode="verbose" From verbose to default: Edit the Istio-override.yaml file’s configuration under the meshConfig section to set enableEnvoyAccessLogService as false: spec: meshConfig: enableTracing: true enableEnvoyAccessLogService: false After the change, re-install Istio again: istioctl install --set profile=demo -f istio-override.yaml After the Istio re-installation, run the following command to set the Amplify Istio Traceability Agent’s mode to “default”: helm repo update helm upgrade --install --namespace apic-control ampc-hybrid axway/ampc-hybrid -f hybrid-override.yaml --set als.mode="default" In default mode the Traceability Agent can be configured to only capture certain request and response headers. By default, we capture all the headers specified in the EnvoyFilter configuration below. See “additional_request_headers_to_log” and “additional_response_headers_to_log” section. apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: patch-gateway-and-sidecars-with-als namespace: <<envoyFilterNamespace>> spec: configPatches: - applyTo: NETWORK_FILTER match: context: ANY listener: filterChain: filter: name: "envoy.filters.network.http_connection_manager" patch: operation: MERGE value: name: "envoy.filters.network.http_connection_manager" typed_config: "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager" access_log: - name: envoy.access_loggers.http_grpc typed_config: "@type": "type.googleapis.com/envoy.extensions.access_loggers.grpc.v3.HttpGrpcAccessLogConfig" additional_request_headers_to_log: ["accept","user-agent","x-envoy-decorator-operation","x-envoy-external-address","x-forwarded-client-cert","x-forwarded-for","x-forwarded-proto","x-request-id","x-b3-parentspanid","x-b3-spanid","x-istio-attributes"] additional_response_headers_to_log: ["connection","content-length","content-md5","content-type","date","etag","date","request-id","response-time","server","start-time","vary"] common_config: filter_state_objects_to_log: ["wasm.upstream_peer","wasm.upstream_peer_id","wasm.downstream_peer","wasm.downstream_peer_id"] log_name: mesh grpc_service: google_grpc: target_uri: ampc-hybrid-als.apic-control.svc.cluster.local:9000 stat_prefix: ampc-hybrid-als To exclude any headers, remove them from “additional_request_headers_to_log” and “additional_response_headers_to_log”. Please note that unless otherwise specified envoyFilterNamespace is “istio-system”. Once the configuration is changed, run the following command: kubectl apply -f <fileName>.yaml Transaction Redaction The Traceability Agent enforces redaction by default. The agent can be configured to show certain paths, query parameters, and header information based on redaction environment variables provided to it. For instructions on how to set the redaction configuration, see Trace Redaction. Once the environment variables are set, put them in a helm override configuration: als: redaction: path: show: ${TRACEABILITY_REDACTION_PATH_SHOW:[]} queryArgument: show: ${TRACEABILITY_REDACTION_QUERYARGUMENT_SHOW:[]} sanitize: ${TRACEABILITY_REDACTION_QUERYARGUMENT_SANITIZE:[]} requestHeader: show: ${TRACEABILITY_REDACTION_REQUESTHEADER_SHOW:[]} sanitize: ${TRACEABILITY_REDACTION_REQUESTHEADER_SANITIZE:[]} responseHeader: show: ${TRACEABILITY_REDACTION_RESPONSEHEADER_SHOW:[]} sanitize: ${TRACEABILITY_REDACTION_RESPONSEHEADER_SANITIZE:[]} Below is a sample redaction configuration: als: redaction: path: show: '[{keyMatch:".*"}]' requestHeader: show: '[{keyMatch:".*"}]' The configuration above will display all URI path information and all request headers. Put your redaction configuration into a file and then execute the following command: helm upgrade --install ampc-hybrid axway/ampc-hybrid --namespace apic-control -f hybrid-override.yaml -f <pathToConfigFile>/config.yaml Monitor whether the Amplify Istio Traceability Agent pods have restarted by executing the following command: kubectl -n <namespace of Amplify Istio Traceability Agent> get pods The deployment of Amplify Istio Traceability Agent will fail if invalid configuration is provided. If there is an error in the pods after executing the command above, you can check the log by executing the following command: kubectl -n <namespace of Amplify Istio Traceability Agent> logs <podName> The logs should display the configuration error. Fix the configuration and repeat the steps above. Last modified March 9, 2022: VIZ-61 rename api observer to business insights (#90) (420cd03) Related Links
Service Mesh Traceability Observe transactions in mesh. 7 minute read Before you begin Before you start, see Deploy your agents with the Axway CLI to learn how to use the CLI to install the Istio agents into your Kubernetes cluster. This page will reference the resources created from the Deploy your agents with the Axway CLI procedure. Prerequisites These prerequisites are required by the Axway Central CLI, which you will use to configure the Amplify Istio Discovery Agent. Node.js >= 10.13.0 and <= 12.14.1 Minimum Axway Central CLI version: 1.7.0 or later For more information, see Install Axway Central CLI. Overview The Amplify Istio Traceability Agent is installed into your Kubernetes cluster as part of deploying the ampc-hybrid helm chart. The Traceability Agent (TA) sends metrics and logs for API activity back to Amplify Central so that you can monitor service activity and troubleshoot your services. The agent publishes a summary of the transaction which can be seen in Business Insights. Once the transaction summary is expanded, you can see all the related spans within a transaction including the request and response headers for each. The Amplify Istio Traceability Agent has two modes; default and verbose. The default mode captures only the headers specified in the EnvoyFilter and the verbose mode captures all the headers in request and response flows. Setup The Amplify Istio Traceability Agent logs and publishes traffic within the Mesh. In order to generate traffic, we need to create certain custom resource definitions (CRDs) in the mesh. Amplify Central resources In order to better filter transactions related to the services in the mesh, certain resources need to be created for each service running on the mesh - namely APIService, APIServiceRevision and APIServiceInstance. The APIService needs to include the attribute “externalAPIID” in its definition. The format of externalAPIID is: clustername-httproutename The “externalAPIID” attribute is used to define the correlation between the Kubernetes cluster name and the http route name on the VirtualService (that is created in the next section) so that the traffic for this APIService can be easily filtered out. The cluster name is the value of the field als.clusterName in the agent override file hybrid-override.yaml generated by the Axway CLI during Istio agent installation to your Kubernetes cluster. Choose a name for the http route name - ideally the same as the name for APIService or very similar to it. Make a note of this route name, as you will need it in the next section. Next, create the following resources in Central using the Axway CLI: Note Create these resources manually only if you are running standalone Amplify Istio Traceability Agent. If you are running Amplify Istio Traceability Agent along with the Discovery agents, these resources are automatically created and you can skip to Istio CRDs. kind: APIService name: <<serviceName>> metadata: scope: kind: Environment name: <<environmentName>> attributes: externalAPIID: <<clustername-http.name>> # http.name to be used for VS spec: {} --- kind: APIServiceRevision name: <<revisionName>> metadata: scope: kind: Environment name: <<environmentName>> spec: apiService: <<serviceName>> definition: type: "oas2" value: <<base64 encoded spec>> --- kind: APIServiceInstance name: <<serviceName>> metadata: scope: kind: Environment name: <<environmentName>> spec: apiServiceRevision: <<revisionName>> endpoint: - host: "apicentral.axway.com" port: 8080 protocol: http routing: basePath: <<basePath>> Here is an example for the Hybrid List demo service: kind: APIService name: mylist metadata: scope: kind: Environment name: meshone attributes: externalAPIID: clusterone-mylist spec: {} --- kind: APIServiceRevision name: list-v1 metadata: scope: kind: Environment name: meshone spec: apiService: mylist definition: type: "oas2" value: eyJzd2FnZ2VyIjoiMi4wIiwiaW5mbyI6eyJ0aXRsZSI6Im15bGlzdCIsImRlc2NyaXB0aW9uIjoiQW4gQVBJIEJ1aWxkZXIgc2VydmljZSIsInZlcnNpb24iOiIxLjAuMCJ9LCJob3N0IjoiMTAwLjk5LjIyMi4xMDI6ODA4MCIsImJhc2VQYXRoIjoiL2FwaSIsInBhdGhzIjp7Ii9saXN0Ijp7ImdldCI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsidHlwZSI6ImFycmF5IiwiaXRlbXMiOnsiJHJlZiI6IiMvZGVmaW5pdGlvbnMvbGlzdCJ9fX19fSwicG9zdCI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsidHlwZSI6Im9iamVjdCIsInByb3BlcnRpZXMiOnt9fX19LCJwYXJhbWV0ZXJzIjpbeyJpbiI6ImJvZHkiLCJuYW1lIjoiYm9keSIsInNjaGVtYSI6eyIkcmVmIjoiIy9kZWZpbml0aW9ucy9saXN0In19XX19LCIvbGlzdC97aWR9Ijp7ImdldCI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsiJHJlZiI6IiMvZGVmaW5pdGlvbnMvbGlzdCJ9fX0sInBhcmFtZXRlcnMiOlt7ImluIjoicGF0aCIsIm5hbWUiOiJpZCIsInR5cGUiOiJzdHJpbmciLCJyZXF1aXJlZCI6dHJ1ZX1dfSwicGFyYW1ldGVycyI6W3sibmFtZSI6ImlkIiwiaW4iOiJwYXRoIiwidHlwZSI6InN0cmluZyIsInJlcXVpcmVkIjp0cnVlfV0sImRlbGV0ZSI6eyJyZXNwb25zZXMiOnsiMjAwIjp7ImRlc2NyaXB0aW9uIjoiT0siLCJzY2hlbWEiOnsidHlwZSI6Im9iamVjdCIsInByb3BlcnRpZXMiOnt9fX19LCJwYXJhbWV0ZXJzIjpbeyJpbiI6InBhdGgiLCJuYW1lIjoiaWQiLCJ0eXBlIjoic3RyaW5nIiwicmVxdWlyZWQiOnRydWV9XX19fSwiZGVmaW5pdGlvbnMiOnsibGlzdCI6eyJ0eXBlIjoib2JqZWN0IiwidGl0bGUiOiJsaXN0IiwicHJvcGVydGllcyI6eyJuYW1lIjp7InR5cGUiOiJzdHJpbmcifSwicHJpY2UiOnsidHlwZSI6InN0cmluZyJ9LCJzdG9yZSI6eyJ0eXBlIjoic3RyaW5nIn19fSwiUmVzcG9uc2VNb2RlbCI6eyJ0eXBlIjoib2JqZWN0IiwicmVxdWlyZWQiOlsic3VjY2VzcyIsInJlcXVlc3QtaWQiXSwiYWRkaXRpb25hbFByb3BlcnRpZXMiOmZhbHNlLCJwcm9wZXJ0aWVzIjp7ImNvZGUiOnsidHlwZSI6ImludGVnZXIiLCJmb3JtYXQiOiJpbnQzMiJ9LCJzdWNjZXNzIjp7InR5cGUiOiJib29sZWFuIiwiZGVmYXVsdCI6ZmFsc2V9LCJyZXF1ZXN0LWlkIjp7InR5cGUiOiJzdHJpbmcifSwibWVzc2FnZSI6eyJ0eXBlIjoic3RyaW5nIn0sInVybCI6eyJ0eXBlIjoic3RyaW5nIn19fSwiRXJyb3JNb2RlbCI6eyJ0eXBlIjoib2JqZWN0IiwicmVxdWlyZWQiOlsibWVzc2FnZSIsImNvZGUiLCJzdWNjZXNzIiwicmVxdWVzdC1pZCJdLCJwcm9wZXJ0aWVzIjp7ImNvZGUiOnsidHlwZSI6ImludGVnZXIiLCJmb3JtYXQiOiJpbnQzMiJ9LCJzdWNjZXNzIjp7InR5cGUiOiJib29sZWFuIiwiZGVmYXVsdCI6ZmFsc2V9LCJyZXF1ZXN0LWlkIjp7InR5cGUiOiJzdHJpbmcifX19fX0= --- kind: APIServiceInstance name: mylist metadata: scope: kind: Environment name: mesh spec: apiServiceRevision: list-v1 endpoint: - host: "apicentral.axway.com" port: 8080 protocol: http routing: basePath: "/mylist" Once configured, use the following command to populate the resources in Amplify Central: axway central apply -f <fileName>.yaml Istio CRDs Gateway First, create a Gateway in the namespace in which the Istio agents were installed. If you already have a Gateway CRD, you can skip to Virtual Service and specify that Gateway in the Virtual Service. In the example below, the selector is specified as the “istio-apic-ingress” (the Ingress gateway that is installed during the Istio installation step in Deploy your agents with the Axway CLI). If you have a separate Ingress gateway that you would like to use, change the spec.selector.istio field to the label of that Ingress gateway instead. Note For more information about Gateway CRD, please refer to Istio documentation. apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: gateway-ingress namespace: <<apic-control>> spec: selector: istio: istio-apic-ingress servers: - hosts: - <<cluster-name>>.hybrid.sandbox.axwaytest.net port: name: <<port-HTTP>> number: <<8080>> protocol: <<HTTP>> For an HTTPS Gateway, use the following configuration: apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: gateway-ingress namespace: <<apic-control>> spec: selector: istio: istio-apic-ingress servers: - hosts: - <<cluster-name>>.hybrid.sandbox.axwaytest.net port: name: port-HTTP number: 443 protocol: HTTPS tls: mode: SIMPLE serverCertificate: /etc/istio/istio-ingressgateway/tls.crt privateKey: /etc/istio/istio-ingressgateway/tls.key Once configured, create the resource using the command: kubectl apply -f <fileName>.yaml Virtual Service Next, create the Virtual Service for the included demo service within the mesh. Unless you have already taken note of the ‘http route name’ part of the externalAPIId attribute associated with the APIService from the previous section, the value for it needs to be extracted from the APIService. First, locate the APIService and make a note of its name and the name of the environment under which it is scoped: axway central get apisvc Then, using this command, print the value for externalAPIID and make a note of the ‘http route name’ part of it: axway central get apisvc <APIService_name> -s <Environment_name> -o yaml | grep externalAPIID Example: axway central get apisvc ✔ Resource(s) successfully retrieved NAME AGE TITLE RESOURCE KIND SCOPE KIND SCOPE NAME RESOURCE GROUP mylist 2 days ago mylist APIService Environment cli-1618415896316 managementaxway central get apisvc mylist -s cli-1618415896316 -o yaml | grep externalAPIID externalAPIID: mycluster-mylist The following example, used to create a VirtualService, applies to the “list” demo service that comes with the Istio agent Helm installation. If you already have a Virtual Service, you can skip to the Pre-existing Virtual Services section below: apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: route-to-list namespace: apic-control spec: hosts: - "*" gateways: - gateway-ingress http: - name: mylist # this should be gotten from the http route name part of the externalAPIId on APIService match: - uri: prefix: /mylist rewrite: uri: /api route: - destination: host: ampc-hybrid-list.apic-demo.svc.cluster.local port: number: 8080 Once configured, create the resource using the command: kubectl apply -f <fileName>.yaml Pre-existing Virtual Service If you have a Virtual Service resource already, simply add a name for (or rename) the http route so that the API Service and the related transactions can be linked in API Central: Example: http: - name: mylist Note The name specified under the http.name field of the VirtualService should be the same as the ‘http route name’ part of the externalAPIId attribute on the APIService. Service Entry If you have an egress hop from a service in the mesh, then create a service entry. See the example below: apiVersion: networking.istio.io/v1beta1 kind: ServiceEntry metadata: name: httpbin.org namespace: apic-control spec: hosts: - httpbin.org ports: - name: http-80 number: 80 protocol: HTTP resolution: DNS The setup is complete for observability in the mesh. To view transactions in Business Insights, generate some traffic for the Hybrid List demo service: curl -v http://demo.sandbox.axwaytest.net:8080/mylist/list Toggling the Traceability Agent After deploying the ampc-hybrid helm chart to your Kubernetes cluster, you can see the Amplify Istio Traceability Agent running. The service is called ampc-hybrid-als. During the step Deploy your agents with the Axway CLI, you were able to select the mode for the Amplify Istio Traceability Agent. If you want to switch the mode, use the following procedure. From default to verbose: Edit the istio-override.yaml file’s configuration under the meshConfig section to set enableEnvoyAccessLogService as true: spec: meshConfig: enableTracing: true enableEnvoyAccessLogService: true After the change, re-install Istio again: istioctl install --set profile=demo -f istio-override.yaml After the Istio re-installation, run the following command to set the Amplify Istio Traceability Agent’s mode to “verbose”: helm repo update helm upgrade --install --namespace apic-control ampchybrid axway/ampc-hybrid -f hybrid-override.yaml --set als.mode="verbose" From verbose to default: Edit the Istio-override.yaml file’s configuration under the meshConfig section to set enableEnvoyAccessLogService as false: spec: meshConfig: enableTracing: true enableEnvoyAccessLogService: false After the change, re-install Istio again: istioctl install --set profile=demo -f istio-override.yaml After the Istio re-installation, run the following command to set the Amplify Istio Traceability Agent’s mode to “default”: helm repo update helm upgrade --install --namespace apic-control ampc-hybrid axway/ampc-hybrid -f hybrid-override.yaml --set als.mode="default" In default mode the Traceability Agent can be configured to only capture certain request and response headers. By default, we capture all the headers specified in the EnvoyFilter configuration below. See “additional_request_headers_to_log” and “additional_response_headers_to_log” section. apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: patch-gateway-and-sidecars-with-als namespace: <<envoyFilterNamespace>> spec: configPatches: - applyTo: NETWORK_FILTER match: context: ANY listener: filterChain: filter: name: "envoy.filters.network.http_connection_manager" patch: operation: MERGE value: name: "envoy.filters.network.http_connection_manager" typed_config: "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager" access_log: - name: envoy.access_loggers.http_grpc typed_config: "@type": "type.googleapis.com/envoy.extensions.access_loggers.grpc.v3.HttpGrpcAccessLogConfig" additional_request_headers_to_log: ["accept","user-agent","x-envoy-decorator-operation","x-envoy-external-address","x-forwarded-client-cert","x-forwarded-for","x-forwarded-proto","x-request-id","x-b3-parentspanid","x-b3-spanid","x-istio-attributes"] additional_response_headers_to_log: ["connection","content-length","content-md5","content-type","date","etag","date","request-id","response-time","server","start-time","vary"] common_config: filter_state_objects_to_log: ["wasm.upstream_peer","wasm.upstream_peer_id","wasm.downstream_peer","wasm.downstream_peer_id"] log_name: mesh grpc_service: google_grpc: target_uri: ampc-hybrid-als.apic-control.svc.cluster.local:9000 stat_prefix: ampc-hybrid-als To exclude any headers, remove them from “additional_request_headers_to_log” and “additional_response_headers_to_log”. Please note that unless otherwise specified envoyFilterNamespace is “istio-system”. Once the configuration is changed, run the following command: kubectl apply -f <fileName>.yaml Transaction Redaction The Traceability Agent enforces redaction by default. The agent can be configured to show certain paths, query parameters, and header information based on redaction environment variables provided to it. For instructions on how to set the redaction configuration, see Trace Redaction. Once the environment variables are set, put them in a helm override configuration: als: redaction: path: show: ${TRACEABILITY_REDACTION_PATH_SHOW:[]} queryArgument: show: ${TRACEABILITY_REDACTION_QUERYARGUMENT_SHOW:[]} sanitize: ${TRACEABILITY_REDACTION_QUERYARGUMENT_SANITIZE:[]} requestHeader: show: ${TRACEABILITY_REDACTION_REQUESTHEADER_SHOW:[]} sanitize: ${TRACEABILITY_REDACTION_REQUESTHEADER_SANITIZE:[]} responseHeader: show: ${TRACEABILITY_REDACTION_RESPONSEHEADER_SHOW:[]} sanitize: ${TRACEABILITY_REDACTION_RESPONSEHEADER_SANITIZE:[]} Below is a sample redaction configuration: als: redaction: path: show: '[{keyMatch:".*"}]' requestHeader: show: '[{keyMatch:".*"}]' The configuration above will display all URI path information and all request headers. Put your redaction configuration into a file and then execute the following command: helm upgrade --install ampc-hybrid axway/ampc-hybrid --namespace apic-control -f hybrid-override.yaml -f <pathToConfigFile>/config.yaml Monitor whether the Amplify Istio Traceability Agent pods have restarted by executing the following command: kubectl -n <namespace of Amplify Istio Traceability Agent> get pods The deployment of Amplify Istio Traceability Agent will fail if invalid configuration is provided. If there is an error in the pods after executing the command above, you can check the log by executing the following command: kubectl -n <namespace of Amplify Istio Traceability Agent> logs <podName> The logs should display the configuration error. Fix the configuration and repeat the steps above. Last modified March 9, 2022: VIZ-61 rename api observer to business insights (#90) (420cd03)