Skip to content
This repository was archived by the owner on Dec 14, 2022. It is now read-only.

Commit c4b1cf7

Browse files
author
Chris Wiechmann
authored
Update README.md
1 parent ce4e5b9 commit c4b1cf7

File tree

1 file changed

+14
-299
lines changed

1 file changed

+14
-299
lines changed

‎elk-traffic-monitor-api/README.md

Lines changed: 14 additions & 299 deletions
Original file line numberDiff line numberDiff line change
@@ -1,304 +1,19 @@
1-
# API-Gateway Traffic-Monitor based on Elastic-Search
1+
# Traffic-Monitor based on Elasticsearch based on API-Builder
22

33
This Axway API-Builder project is exposing the API-Gateway Traffic-Monitor API based on an Elasticsearch cluster to improve query performance on large datasets.
44

55
For more information on using API Builder please see the [API Builder Getting Started Guide](https://docs.axway.com/bundle/API_Builder_4x_allOS_en/page/api_builder_getting_started_guide.html).
66

7-
## Running your service
8-
9-
**Run and develop your service**
10-
11-
```bash
12-
npm start
13-
```
14-
15-
## Service configuration
16-
17-
On startup, the service will load and merge all configuration files named `conf/*default.js` or `conf/*local.js`. If a property is configured in both `default.js` and `local.js`, the `local.js` value will take precedence. Configuration files are applied alphabetically "default" first, and then "local", so that any setting in "local" files, will take precedence over any previously set in "default".
18-
19-
### Local configuration
20-
21-
The local configuration files are explcitly ignored by git, npm, and docker. Any sensitive keys should be applied to local configuration.
22-
23-
### Runtime configuration
24-
25-
If you need to be able to configure a runtime setting, then you can expose the desired properties with environment variables before running the service. For example, if you want to be able to configure the apikey when your service runs, then modify an appropriate configuration file, e.g. `conf/local.js`, so that apikey will use the `process.env.myapikey` environment variable:
26-
27-
```js
28-
// local.js
29-
module.exports = {
30-
apikey: process.env.myapikey
31-
}
32-
```
33-
34-
Then you would supply the variable at runtime using `bash`:
35-
36-
```bash
37-
myapikey=secret npm start
38-
```
39-
40-
Supplying the runtime using Windows:
41-
42-
```
43-
SET myapikey=secret && npm start
44-
```
45-
46-
## Additional custom configuration
47-
48-
Additional environment variables may be necessary depending on how your service was developed. For example you may have a flow that is configured in advance to consume the environment variable `SECRETKEY`. Your flow would access it at `$.env.SECRETKEY` and you would set the key at runtime. For example, using `bash`:
49-
50-
```bash
51-
SECRETKEY=foobar npm start
52-
```
53-
54-
Using Windows:
55-
56-
```
57-
SET SECRETKEY=foobar && npm start
58-
```
59-
60-
## Invoking an API locally
61-
62-
This makes a request against the custom run above:
63-
64-
```bash
65-
curl -X GET -u <yourKey>: "http://127.0.0.1:8080/api/greet?username=user"
66-
```
67-
68-
## Logs
69-
The service streams its log messages to the console, this allows for easier integration with third-party log aggregation tools. For additional information on integrating API Builder logging with a third-party log aggregation tool, refer to [Log aggregation in API Builder](https://docs.axway.com/bundle/API_Builder_4x_allOS_en/page/export_api_builder_logs_into_a_data_store.html).
70-
71-
### Logs format
72-
The API Builder logs use different logging utility.
73-
At runtime your service will log all messages to the console.
74-
The Request/Response Logs have a format:
75-
```
76-
<timestamp> [requiest-id: <requestid>] (Response|Request) <json data>
77-
```
78-
79-
### Set __logLevel__
80-
Edit `./conf/default.js` from the root of your project.
81-
Set the log level by changing `logLevel` to one of the accepted values: __debug__, __error__, __fatal__, __info__, __trace__, or __warn__.
82-
```js
83-
// logging configuration
84-
logLevel: 'debug', // Log level of the main logger.
85-
```
86-
87-
# Dockerise API Builder Service
88-
89-
We already know that generation of API Builder service is very easy with the help of the API Builder CLI tool.
90-
However, one might want to run it within Docker container so it can later be weaved as part of a more complex solution. This is a step by step tutorial on how to run API Builder service within a Docker container.
91-
92-
## Prerequisites
93-
94-
Here are the technical requirements for being able to execute the steps suggested in this guide. You need to have the following installed:
95-
* Docker - The installation of Docker is via dedicated installer for specific operation system. [Read the official guide for how to install Docker](https://docs.docker.com/install/).
96-
* API Builder CLI - It is a node module published in [npm public repository](https://www.npmjs.com/package/@axway/api-builder). Basically, it is a one line command as follows:
97-
98-
99-
```npm install -g @axway/api-builder```
100-
101-
[You can read the official documentation for more information.](https://docs.axway.com/bundle/API_Builder_4x_allOS_en/page/api_builder_getting_started_guide.html)
102-
103-
## Step 1: Scaffold and Run API Builder Service
104-
105-
If you already have a generated service you can go to Step 2.
106-
Still for the completeness of this tutorial we are showing you how to do that:
107-
108-
```
109-
api-builder init <YOUR_APP_NAME>
110-
cd <YOUR_APP_NAME>
111-
npm install --no-optional
112-
npm start
113-
```
114-
115-
Once your service is running, you can point your browser at http://localhost:8080/console to access the API Builder user interface (UI) console.
116-
[You can read the official documentation of how to further use the API Builder Console.](https://docs.axway.com/bundle/API_Builder_4x_allOS_en/page/api_builder_getting_started_guide.html)
117-
118-
Excellent! Now you tested out how your service is running directly on your machine.
119-
Now please stop the service with using Ctrl + C in your terminal where the service is running and go to the next step.
120-
121-
**Variable placeholders used in this section**
122-
123-
Variable Placeholder | Description
124-
-------------- | -------------
125-
<SERVICE_NAME> | This is the service name of your choice e.g. "myservice" that the API Builder CLI is using. It will create folder with the same name and will place all the generated files in that folder.
126-
127-
128-
## Step 2: Prepare to run your service within Docker
129-
In Step 1 we showed the classic way of running the service locally. Let's see how to run the service within a container. When running your service in a container it is often desirable to have the configuration (or parts of it) set at runtime rather than relying on static values. To achieve this one could update the configuration files to read from the environment. In Node.js world this is done with reading variables from process.env e.g. process.env.<VARIABLE_NAME>.
130-
131-
### Configuration Files Types
132-
133-
The configuration files that could contain environment variables are placed in <SERVICE_FOLDER>/conf folder.
134-
All the variables in your configuration files that support to be taken from "process.env.<VARIABLE_NAME>" could be provided when running the Docker container. Bellow are presented different configuration files, their location, and their example content. Note that this tutorial does not require connector installation. The connector configuration is just shown to inform you that you will have additional set of environment variables that must be provided when dockerising API Builder service with connectors.
135-
136-
Configuration File | Location | Example
137-
-------------- | ------------- | -------------
138-
Service Configuration | `<SERVICE_FOLDER>/conf/default.js` | ```module.exports = { apiKey: process.env.APIKEY };```
139-
Connector Configuration | Example with MySQL. The file will be named: <SERVICE_FOLDER>/conf/mysql.default.js | ```module.exports = {connectors: { mysql: { connector: '@axway/api-builder-plugin-dc-mysql' connectionPooling: true, connectionLimit: 10, host: process,env.MYSQL_HOST || 'localhost', port: 3306, database: 'mysql', user: process.env.MYSQL_USER, password: process.env.MYSQL_PASSWORD, generateModelsFromSchema: true, modelAutogen: false}}};```
140-
141-
Excellent! Now you defined which variables could be provided at runtime while starting your Docker container.
142-
Let's see next how to prepare your Docker image and run Docker container out of it.
143-
144-
## Step 3: Create Docker image for your service - meet the Dockerfile
145-
146-
API Builder Services come with a Dockerfile. When we generate a service with API Builder CLI we get a service that has Dockerfile in its root.
147-
This file is just one possible Dockerfile that can be used for Docker image creation. One can perfectly create another Dockerfile tailored for specific needs. The sample Dockerfile content is:
148-
149-
```
150-
# This line defines which node.js Docker image to leverage
151-
# Available versions are described at https://hub.docker.com/_/node/
152-
FROM node:8-alpine
153-
154-
# Sets the default working directory to /app which is where we've copied the service files to.
155-
WORKDIR /app
156-
157-
# TODO: for security purposes, you should update this Dockerfile to specify your own target user/group
158-
# -S stands for '--system'
159-
# -G stands for group
160-
# -R changes the ownership rights of a file recursively
161-
RUN addgroup -S axway-group && adduser -S axway-user -G axway-group && \
162-
chown -R axway-user:axway-group /app
163-
164-
# Set non-root user
165-
USER axway-user
166-
167-
# Denotes to copy all files in the service to 'app' folder in the container
168-
COPY . /app
169-
170-
# Install service dependencies relevant for production builds skipping all development dependencies.
171-
RUN npm install --production --no-optional
172-
173-
# Starts the service
174-
CMD ["node", "."]
175-
```
176-
177-
> In case that you want to specify your own user and password, you could use `ENV` (Environment Variables) in tha APP Dockerfile, please find an example below:
178-
179-
```
180-
# Set specific user & password using Env Vars
181-
ENV NONROOT_USER="<user>"
182-
183-
RUN adduser $NONROOT_USER && \
184-
echo $NONROOT_USER":<password>" | chpasswd
185-
186-
USER $NONROOT_USER
187-
```
188-
189-
Docker image can be created with:
190-
191-
```docker build -t <IMAGE_NAME> ./```
192-
193-
Now we are able to run as much containers as we want out of this image. To check the presence of the newly created image one can run:
194-
195-
```docker image ls```
196-
197-
Docker image with name <IMAGE_NAME> should be visible in the list with images.
198-
Usually those images are published in Docker repository but how to do this is out of scope for this tutorial.
199-
[See `docker push` documentation](https://docs.docker.com/engine/reference/commandline/push/)for information on how to push your image to a docker registry.
200-
201-
**Variable placeholders used in this section**
202-
203-
Variable Placeholder | Description
204-
-------------- | -------------
205-
<IMAGE_NAME> | This is the docker image name of your choice e.g. "myimage". Docker is using this name to create Docker image for the service with it. After creation of the image using 'docker image ls' should show an image with such name in the list with all available images on your machine.
206-
207-
## Step 4: Run the Docker Container and Access the Service
208-
Things to observe when running your service within Docker container:
209-
210-
* **Set Environment Variables:** Make sure that you provide values on all the environment variables that have no default values as well as on those you have to change the default values. Make sure you provide values not only for service configuration but also for all the connectors you have installed in your service.
211-
* **Production Deployment:** As the image is built in production mode, the service is no longer editable. The API Builder console will not be available.
212-
* **Run Multiple Services:** Having containerized your service it is now possible, and generally desirable, to run multiple instances of your service. This allows you to address load balancing, scalablity, and the resilience of your service.
213-
214-
Next we will show two approaches of running the docker container. Pick up the first one if you don't want to map ports on you host machine and the second one if you want to access the service like if it is working on your localhost.
215-
216-
#### Approach 1: Running Docker Container - Plain
217-
Using the image created on Step 3 you can now run a Docker container. Running Docker container will execute the CMD specified in the Dockefile as a result of which your service will be started.
218-
```
219-
# Run the container exposing the port 8080 to your host so the service is accessible on that port
220-
docker run --name <CONTAINER_NAME> <IMAGE_NAME>
221-
```
222-
223-
#### How to provide environment variables for Docker container?
224-
225-
In case you want to run the service within Docker container providing runtime values for those properties that are configured to be read from environment (see Step 2) one could do as follows:
226-
227-
```
228-
# Run the container setting the port where the service will run within the container to 8081
229-
docker run --name <CONTAINER_NAME> -e PORT=8081 <IMAGE_NAME>
230-
```
231-
232-
#### Other useful commands
233-
```
234-
# If you want to see that your container is running
235-
# You should see <CONTAINER_NAME> in the list of started containers
236-
docker container ls
237-
238-
# Once ran the running container could be stopped with
239-
docker container stop <CONTAINER_NAME>
240-
241-
# ... and started again with
242-
docker container start <CONTAINER_NAME>
243-
```
244-
245-
#### How to access the service?
246-
247-
First of all you need to check the IP Address that Docker is using to run your container:
248-
```docker inspect <CONTAINER_NAME> | grep '"IPAddress"' | head -n 1```
249-
250-
Now check the value of your apikey property in `<SERVICE_FOLDER>/conf/default.js.`
251-
252-
Using that apikey you could execute:
253-
254-
```
255-
curl -X GET -u <API_KEY>: "http://<IP_ADDRESS>:<PORT>/api/greet?username=APIBuilder"
256-
```
257-
258-
**Variable placeholders used in this section**
259-
260-
Variable Placeholder | Description
261-
-------------- | -------------
262-
<IMAGE_NAME> | This is the name of the docker image name specified by you on the previous step.
263-
<CONTAINER_NAME> | This is the docker container name of your choice e.g. "mycontainer". Docker will use this name to run container based on the Docker image specified with <IMAGE_NAME>.
264-
<API_KEY> | Must be replaced with the value of the apiKey property specified in `<SERVICE_FOLDER>/conf/default.js`
265-
< PORT > | 8080 if not overiden
266-
<IP_ADDRESS> | Should be replaced with the value returned by the command that grep the IP Address shown above
267-
268-
269-
270-
#### Approach 2: Running Docker Container - Detached Mode with Port Mapping
271-
Using the image we can run a Docker container with:
272-
```
273-
# Run the container exposing the port 8080 to your host so the service is accessible on that port
274-
docker run --name <CONTAINER_NAME> -p 8080:8080 -d <IMAGE_NAME>
275-
276-
# Now access with
277-
curl -X GET -u <API_KEY>: "http://localhost:8080/api/greet?username=APIBuilder"
278-
```
279-
280-
Alternatively with overriding the port we can do
281-
```
282-
# Note that if you provide another port with -e option you must map that provided port value to the desired port number of the host machine
283-
# The example bellow run the service within container on 8081 and map that port to 8082 of the host machine
284-
docker run --name <CONTAINER_NAME> -e PORT=8081 -p 8082:8081 -d <IMAGE_NAME>
285-
286-
287-
# Now access with
288-
curl -X GET -u <API_KEY>: "http://localhost:8082/api/greet?username=APIBuilder"
289-
```
290-
291-
**Docker options explained**
292-
293-
Variable Placeholder | Description
294-
-------------- | -------------
295-
-d | Run container in background and print container ID
296-
-p | The -p option allows you to map ports on your host machine to ports inside the container e.g. <host_port>:<container_port>
297-
298-
**Variable placeholders used in this section**
299-
300-
Variable Placeholder | Description
301-
-------------- | -------------
302-
<IMAGE_NAME> | This is the name of the docker image name specified by you on the previous step.
303-
<CONTAINER_NAME> | This is the docker container name of your choice e.g. "mycontainer". Docker will use this name to run container based on the Docker image specified with <IMAGE_NAME>.
304-
<API_KEY> | Must be replaced with the value of the apiKey property specified in `<SERVICE_FOLDER>/conf/default.js`
7+
## How it works?
8+
The Axway API-Gateway Traffic-Monitor is normally communicating with the REST-API exposed the Admin-Node-Manager, which is using as a ultimate backbone the integrated OPSDB running on each API-Gateway instance. This API-Builder project is exposing a few methods (for instance the search request) of the same API, but implements it using the Elasticsearch cluster. It becomes basically a middleware or mediation layer between the requesting client and Elasticsearch.
9+
10+
To implement the required API an API-First approach has been used:
11+
1. Defined the subset of API-methods that are needed
12+
2. Imported the API-Definition into the API-Builder project
13+
3. Imported the [Elasticsearch connector]((https://github.com/Axway-API-Builder-Ext/api-builder-extras/tree/master/api-builder-plugin-fn-elasticsearch)) into the API-Builder project
14+
4. Created a flow
15+
- that handles all filter parameters
16+
- Query Elasticsearch
17+
- Format and return the response in the way it is expected
18+
19+
Documenting this API-Builder project is still in progress

0 commit comments

Comments
 (0)