Microservice Mercadolibre Users AWS

Microservice for user management exemplifying part of the ML development architecture, implemented with Systems Manager Parameter Store, Api-Gateway, Serverless-Framework, Lambda, NodeJs, Sequelize, Mysql, Docker, Docker-compose, Amazon RDS, Unit Test with Jest, among others. AWS services are tested locally. The project code and its documentation (less technical doc) have been developed in English.

• Api Doc ML Users
• Playlist functionality test

Index 📜

View

Section 1) Description, configuration and technologies.

• 1.0) Project Description.
• 1.1) Project Execution.
• 1.2) Project Configuration from Scratch
• 1.3) Docker Setup and Database Migration
• 1.4) Technologies.

Section 2) Endpoints and Examples

• 2.0) EndPoints and resources.
• 2.1) Examples.

Section 3) Functionality Test and References

• 3.0) Functionality Test.
• 3.1) References.

Section 1) Description, configuration and technologies.

1.0) Project Description 🔝

See

1.0.0) General description

• The Microservice is designed under the MVC architecture. This architecture consists of and is divided into the model layer (definition of the user table), the service layer (the connection and transactions to the db with sequelize) and the controller layer (the implemented lambdas).
• Each lambda performs the token authentication check, those that wait for a body type event check these fields and all the logic to be performed is abstracted from it to decouple functionalities together with low coupling.
• Endpoints that allow the return of more than one object according to the applied search logic are handled with pagination if required. Default pagination is applied.

1.0.1) Description Architecture and Operation

• The image of the AWS architecture used describes the operating flow of the microservice in a general way. Any request to the microservice starts from a client (Postman, server, etc.).
• Step 0: This request is received by the api-gateway and will only be validated if the correct x-api-key is found within the headers of said request.
• Steps 1A, 1B, etc: All these steps correspond to an endpoint with its specific resource. For ex. for getAllUsers (1A) it is http://localhost:4000/dev/users/list ....check those endpoints in endpoints section. Each lambda performs x-api-key and token checking.
• Steps 2: The lambdas perform the validations of the corresponding ssm with the System Manager Paramater Store... they validate token, connection values with the db, etc.
• Steps 3: The lambdas perform the necessary transactions and operations with the db (Mysql).
• Clarifications: This operation is emulated within the same network and in a local environment with the corresponding serverless plugins.

1.1) Project Execution 🔝

See

• Once a work environment has been created through some IDE, we clone the project

git clone https://github.com/andresWeitzel/Microservice_Mercadolibre_Users_AWS

• We position ourselves on the project

cd 'projectName'

• We install the LTS version of Nodejs(v18)
• We install the Serverless Framework globally if we have not already done so

npm install -g serverless

• We verify the version of Serverless installed

sls -v

• We install all the necessary packages

npm i

• Make sure Docker are installed on your system (for Windows, use Docker Desktop)

• Start and build the MySQL database container:

docker-compose up -d

• Verify the container is running:

docker ps

• If you need to reset the database:

docker-compose down -v
docker-compose up -d

• To view database logs:

docker-compose logs mysql

• To access MySQL directly:

docker exec -it mercadolibre_users_mysql mysql -u mercadolibre_user -p

• The ssm and env variables used in the project are maintained to simplify the project configuration process. It is recommended to add the corresponding files (serverless_ssm.yml and .env) to the .gitignore.

• The start script configured in the project's package.json is responsible for launching

• The serverless-offline plugin

• The remark-lint plugin for .md files (only --output is applied for check and autoformat without terminating the process and being able to execute the serverless script)

• We run the app from terminal.

npm start

• If a message appears indicating that port 4000 is already in use, we can terminate all dependent processes and run the app again

npx kill-port 4000
npm start

1.2) Project Configuration from Scratch 🔝

Ver

• We create a work environment through some ide, after creating a folder we position ourselves on it

cd 'projectName'

• We install the latest LTS version of Nodejs(v18)
• We install the Serverless Framework globally if we have not already done so

npm install -g serverless

• We verify the version of Serverless installed

sls -v

• Make sure Docker are installed on your system (for Windows, use Docker Desktop)

• Start and build the MySQL database container:

docker-compose up -d

• Verify the container is running:

docker ps

• If you need to reset the database:

docker-compose down -v
docker-compose up -d

• To view database logs:

docker-compose logs mysql

• To access MySQL directly:

docker exec -it mercadolibre_users_mysql mysql -u mercadolibre_user -p

1.3) Docker Setup and Database Migration 🔝

See

Database Setup with Docker

The project uses Docker to manage the MySQL database. Here's how to set it up:

1. Make sure you have Docker and Docker Compose installed on your system.

2. The database configuration is defined in docker-compose.yml:

version: '3.8'
services:
  mysql:
    image: mysql:8.0
    container_name: mercadolibre_users_mysql
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: microdb_mercadolibre
      MYSQL_USER: mercadolibre_user
      MYSQL_PASSWORD: mercadolibre_pass
    ports:
      - "3306:3306"
    volumes:
      - mysql_data:/var/lib/mysql
      - ./init:/docker-entrypoint-initdb.d

3. Database initialization scripts are located in the init directory:
   • 01_microdb_mercadolibre_DDL.sql: Creates database and tables
   • 02_microdb_mercadolibre_DML_INSERTS.sql: Inserts initial data

Docker Commands

Here are the essential Docker commands for managing the database:

# Start the database container
docker-compose up -d

# Stop the container
docker-compose down

# Stop and remove volumes (to reset database)
docker-compose down -v

# View container logs
docker-compose logs mysql

# Access MySQL container
docker exec -it mercadolibre_users_mysql mysql -u mercadolibre_user -p

Database Reset

If you need to reset the database to its initial state:

1. Stop the container and remove volumes:

docker-compose down -v

2. Recreate the container:

docker-compose up -d

This will:

• Create a fresh database
• Execute all DDL scripts to create tables
• Insert initial data from DML scripts

Sample Data

The database comes pre-populated with sample data including:

• Users and user details
• Addresses and address details
• Sellers information
• Products and product details

Example of user data:

INSERT INTO users (id, nickname, first_name, last_name, email, identification_type, identification_number, country_id, creation_date, update_date) VALUES
(1, 'RAFA-CON', 'Rafael', 'Castro', '[email protected]', 'DNI', '445938822', 'AR', NOW(), NOW()),
(2, 'JAVIER GONZALEZ', 'Javier', 'Gonzalez', '[email protected]', 'DNI', '2672268765', 'AR', NOW(), NOW());

Example of product data:

INSERT INTO products (id, site_id, title, subtitle, seller_id, category_id, official_store_id, price, base_price, original_price, initial_quantity, available_quantity, creation_date, update_date) VALUES
(1, 'MLA', 'iPhone 13 Pro Max 256GB', 'Nuevo, sellado, con garantía oficial de Apple', 1, 'MLA1055', NULL, 1200000.00, 1200000.00, 1200000.00, 10, 8, NOW(), NOW()),
(2, 'MLA', 'Samsung Galaxy S21 Ultra', 'Último modelo, libre de fábrica', 2, 'MLA1055', NULL, 950000.00, 950000.00, 950000.00, 15, 12, NOW(), NOW());

1.4) Technologies 🔝

See

| Technologies | Version | Purpose |
| ------------- | ------------- | ------------- | | SDK | 4.3.2 | Automatic Module Injection for Lambdas | | Serverless Framework Core v3 | 3.23.0 | Core Services AWS | | Systems Manager Parameter Store (SSM) | 3.0 | Management of Environment Variables | | Jest | 29.7 | Framework para pruebas unitarias, integración, etc. | | Amazon Api Gateway | 2.0 | API Manager, Authentication, Control and Processing | | NodeJS | 14.18.1 | js library | | Sequelize | ^6.11.0 | ORM | | Mysql | 10.1 | SGDB | | XAMPP | 3.2.2 | Server package | | VSC | 1.72.2 | IDE | | Postman | 10.11 | http client | | CMD | 10 | Símbolo del Sistema para linea de comandos | | Git | 2.29.1 | Version control | | Otros | Otros |

Plugin
Serverless Plugin
serverless-offline
serverless-offline-ssm

| Extensión |
| ------------- | | Prettier - Code formatter | | YAML - Autoformatter .yml | | Error Lens - for errors and indent | | Tabnine - IA Code | | Otros - Otros |

Section 2) Endpoints and Examples.

2.0) Endpoints and resources 🔝

See

GET type operations:

• http://localhost:4000/dev/v1/test

• http://localhost:4000/dev/v1/db-connection

• http://localhost:4000/dev/v1/users/list

• http://localhost:4000/dev/v1/users/id/{user-id}

• http://localhost:4000/dev/v1/users/country-id/{country-id}

• http://localhost:4000/dev/v1/users/email/{email}

• http://localhost:4000/dev/v1/users/first-name/{first-name}

• http://localhost:4000/dev/v1/users/identification-number/{ident-number}

• http://localhost:4000/dev/v1/users/identification-type/{ident-type}

• http://localhost:4000/dev/v1/users/last-name/{last-name}

• http://localhost:4000/dev/v1/users/nickname/{nickname}

• http://localhost:4000/dev/v1/users/creation-date/{creation-date}

• http://localhost:4000/dev/v1/users/update-date/{update-date}

• All endpoints are optional paginated except /test, /db-connection and /id/{user-id}}

POST type operations:

• http://localhost:4000/dev/v1/users/add-user/

PUT type operations:

• http://localhost:4000/dev/v1/users/update-user/{user-id}

DELETE type operations:

• http://localhost:4000/dev/v1/users/delete-user/{user-id}

Clarifications

• {required-value}
• Default pagination: ?page=0&limit=5
• Optional pagination: ?page={nro}&limit={nro}

2.1) Examples 🔝

See

2.1.0) Variables in Postman

Variable	Initial value	Current value
base_url	http://localhost:4000/dev/	http://localhost:4000/dev/
x-api-key	f98d8cd98h73s204e3456998ecl9427j	f98d8cd98h73s204e3456998ecl9427j
bearer_token	Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c	Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

2.1.1) GET type operations

Database connection

Request (GET) | Code Snippet

curl --location 'http://localhost:4000/dev/v1/db-connection' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data ''

Response (200 OK)

{
    "message": "Connection has been established successfully."
}

Response (400 Bad Request)

{
    "message": "Bad request, check missing or malformed headers"
}

Response (401 Unauthorized)

{
    "message": "Not authenticated, check x_api_key and Authorization"
}

Response (500 Internal Server Error)

{
    "message": "Error in connection lambda. Caused by Error: throw a new error to check for the exception caught by lambda"
}

Get Paged Users

Request (GET) | Code Snippet

curl --location 'http://localhost:4000/dev/v1/users/list?page=0&limit=2' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data ''

Response (200 OK)

{
    "message": [
        {
            "id": 3,
            "nickname": "HECTOR SS G",
            "first_name": "Hector",
            "last_name": "Gomez",
            "email": "[email protected]",
            "identification_type": "DNI",
            "identification_number": "2172265827",
            "country_id": "AR",
            "creation_date": "2023-03-20 21:02:33",
            "update_date": "2023-03-20 21:02:33"
        },
        {
            "id": 4,
            "nickname": "GABRIELA JIMENEZ",
            "first_name": "Gabriela",
            "last_name": "Jimenez",
            "email": "[email protected]",
            "identification_type": "DNI",
            "identification_number": "410871223",
            "country_id": "AR",
            "creation_date": "2023-03-20 21:02:33",
            "update_date": "2023-03-20 21:02:33"
        }
    ]
}

Response (400 Bad Request)

{
    "message": "Bad request, check missing or malformed headers"
}

Response (401 Unauthorized)

{
    "message": "Not authenticated, check x_api_key and Authorization"
}

Response (500 Internal Server Error)

{
    "message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}

2.1.2) POST type operations

Add a User

Request (POST) | Code Snippet

curl --location 'http://localhost:4000/dev/v1/users/add-user/' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data-raw '{
            "nickname": "VALE18BNX",
            "first_name": "Valeria",
            "last_name": "Castro",
            "email": "[email protected]",
            "identification_type": "DNI",
            "identification_number": "3987261233",
            "country_id": "AR12"
        }'

Response (200 OK)

{
    "message": {
        "id": null,
        "nickname": "VALE18BNX",
        "first_name": "Valeria",
        "last_name": "Castro",
        "email": "[email protected]",
        "identification_type": "DNI",
        "identification_number": "3987261233",
        "country_id": "AR12",
        "creation_date": "2023-06-28T16:46:31.000Z",
        "update_date": "2023-06-28T16:46:31.000Z"
    }
}

Response (400 Bad Request)

{
    "message": "Bad request, check missing or malformed headers"
}

Response (401 Unauthorized)

{
    "message": "Not authenticated, check x_api_key and Authorization"
}

Response (500 Internal Server Error)

{
    "message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}

2.1.3) PUT type operations

Edit a User

Request (PUT) | Code Snippet

curl --location --request PUT 'http://localhost:4000/dev/v1/users/update-user/26' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data-raw '{
            "nickname": "VALE18BNX EDITED",
            "first_name": "Valeria EDITED",
            "last_name": "Castro",
            "email": "[email protected]",
            "identification_type": "DNI",
            "identification_number": "3987261233",
            "country_id": "AR12",
            "creation_date": "2023-06-28 16:46:31",
            "update_date": "2023-06-28 16:46:31"
        }'

Response (200 OK)

{
    "message": {
        "id": 26,
        "nickname": "VALE18BNX EDITED",
        "first_name": "Valeria EDITED",
        "last_name": "Castro",
        "email": "[email protected]",
        "identification_type": "DNI",
        "identification_number": "3987261233",
        "country_id": "AR12",
        "creation_date": "2023-06-28 19:46:31",
        "update_date": "2023-06-28 16:53:17"
    }
}

Response (400 Bad Request)

{
    "message": "Bad request, check missing or malformed headers"
}

Response (401 Unauthorized)

{
    "message": "Not authenticated, check x_api_key and Authorization"
}

Response (500 Internal Server Error)

{
    "message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}

2.1.4) DELETE type operations

Delete a User

Request (DELETE) | Code Snippet

curl --location --request DELETE 'http://localhost:4000/dev/v1/users/delete-user/26' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data ''

Response (200 OK)

{
    "message": "User has been deleted successfully."
}

Response (400 Bad Request)

{
    "message": "Bad request, check missing or malformed headers"
}

Response (401 Unauthorized)

{
    "message": "Not authenticated, check x_api_key and Authorization"
}

Response (500 Internal Server Error)

{
    "message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}

Section 3) Functionality Testing and References.

3.0) Functionality Test 🔝

See

Types of operations | See

3.1) References 🔝

See

AWS Services and Tools

• AWS Lambda Documentation
• API Gateway Best Practices
• Systems Manager Parameter Store
• Amazon RDS Documentation
• AWS CloudWatch Logs
• AWS IAM Best Practices

Serverless Framework

• Serverless Framework Documentation
• Serverless Framework Plugins
• Serverless Offline Plugin
• Serverless SSM Plugin
• Serverless OpenAPI Documentation
• Serverless Auto Swagger

Database and ORM

• Sequelize Documentation
• MySQL Documentation
• Docker MySQL Image
• Docker Compose Documentation
• Sequelize Migrations
• Sequelize Associations

Testing and Development

• Jest Documentation
• Node.js Documentation
• Postman Documentation
• VS Code Documentation
• Git Documentation
• Docker Desktop Documentation

API Design and Best Practices

• REST API Best Practices
• API Security Best Practices
• OpenAPI Specification
• API Documentation Best Practices
• HTTP Status Codes

Mercadolibre API

• Mercadolibre API Documentation
• Mercadolibre Users API
• Mercadolibre Products API
• Mercadolibre Authentication

Development Tools and Resources

• AWS Design Tool (draw.io)
• Postman Collection Examples
• VS Code Extensions for AWS
• Docker Hub
• GitHub Actions
• Node.js Best Practices

Community and Learning Resources

• AWS Community Builders
• Serverless Framework Forum
• Stack Overflow
• AWS YouTube Channel
• Serverless Framework YouTube
• Mercadolibre Developers Blog