Running Amazon DynamoDB locally

Since DynamoDB is a cloud no sql DB, there are limited ways to run it locally on premise. In this article, we will discuss using a local dynamoDB instance so you do not incur unnecessary costs on the cloud. This can be used for running tests or local development.

Running DynamoDB Locally

AWS DynamoDB is a serverless no sql cloud database. It is a little different to use offline, so one simple way to create an offline instance of it, is to use the DynamoDB Docker image available by Amazon on Docker Hub. This way we will not incur costs on the cloud when we are running our tests.

The quickest way to do this is to install Docker Desktop on your computer, https://docs.docker.com/get-docker/

Then once it is installed, and running. Pull the Amazon DynamoDB docker image from Docker hub: https://hub.docker.com/r/amazon/dynamodb-local

docker pull amazon/dynamodb-local

Then you can run this docker image at any time using the docker run command. It will be available on localhost:8000. Make sure the docker container is running when accessing it.

docker run -p 8000:8000 amazon/dynamodb-local

Accessing a local DynamoDB from Node.js: Setup using AWS-SDK

Assuming you already have a node.js already project set up. You should also already have an AWS Account and be familiar with the AWS-SDK.

• Environment variables containing AWS access keys, and secret access keys, region, port, Database Table names, etc

Now you would like to connect to the local instance of DynamoDB instead of the cloud one, for local testing.

To do this you need to install the Amazon SDK in your node project. You might already have it installed if you were using the cloud database or any other AWS services.

https://www.npmjs.com/package/aws-sdk

npm i aws-sdk

You’ll also need to create a file for your AWS Configuration, it can be named as:

aws-config.js

Inside this file, you can include the table name, and your access keys.

module.exports = {
    aws_table_name: process.env.DYNAMODB_TABLENAME,
    aws_local_config: {
      //Provide details for local configuration
    },
    aws_remote_config: {
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
      region: process.env.AWS_DEFAULT_REGION,
    }
};

Inside your environment variables file, you should add the following:

DYNAMODB_LOCAL_ADDRESS=http://localhost:8000
DYNAMODB_TABLENAME=

Once that is all done, we are ready to set up a connection to the local DB and feed it some data, create some tables. etc.

Every time we stop and start the docker container, it will lose all its data, so we need to call this function to set up the tables every time the server starts. Please do not save any important information in this table as it will not be persisted.

Let’s create a file to set up a local db, so can import it into our main index.js

local-db.js

Inside this file we just created we will add the following:

• require the aws-sdk
• import some mock data to insert into our table (optional)
• define a table using the DynamoDB schema, passing it the table name and the primary key (id)

const AWS = require('aws-sdk');
const awsConfig = require('./aws-config');
const data = require('../tests/mockData/data.json');
const table = {
    TableName : process.env.DYNAMODB_TABLENAME,
    KeySchema: [
        { AttributeName: "id", KeyType: "HASH"},
    ],
    AttributeDefinitions: [
        { AttributeName: "id", AttributeType: "S" },
    ],
    ProvisionedThroughput: {
        ReadCapacityUnits: 5,
        WriteCapacityUnits: 5
    }
};

• define a function to create this table and insert some data into it
• We will perform async operations, therefore this function will need to be labelled as async
• We use the dynamodb.createTable function
• If the tables already exist, we do not create them again
• (Optional) We also feed in dummy data if we need it, using the dynamodb.put — Using a loop here to insert multiple records.

const createDynamoDBLocal = async() => {
    ("Setting up local dynamo db.. make sure docker container is running.")
    AWS.config.update({
        region: process.env.AWS_DEFAULT_REGION,
        endpoint: process.env.DYNAMODB_LOCAL_ADDRESS,
      });

// create table    
    const dynamodb = new AWS.DynamoDB();
try {
        const data = await dynamodb.listTables({Limit: 10}).promise();
if (data.TableNames.length >= 1) {
            console.log ("Existing docker container already running.")
} else {
            await dynamodb.createTable(table).promise();

// optional to add dummy data 
const client = new AWS.DynamoDB.DocumentClient();
const params = {
            TableName: process.env.DYNAMODB_TABLENAME,
            Item: data
        };
await client.put(params).promise();
params.TableName = process.env.DYNAMODB_TABLENAME;
        
   data.forEach( async(item) => {
            params.TableName = process.env.DYNAMODB_TABLENAME;
            params.Item = item;
            await client.put(params).promise();
        });
    } catch (err) {
        console.log (err)
    }
}

• Then we will create another function in this file which just returns an instance of the DynamoDB client, so we can use it in our app. The DocumentClient makes it easier to interact with our DB.

const getDynamoDBLocal = () => {
    AWS.config.update({
        region: process.env.AWS_DEFAULT_REGION,
        endpoint: process.env.DYNAMODB_LOCAL_ADDRESS,
    });
const dynamoClient = new AWS.DynamoDB.DocumentClient();     
    return dynamoClient; 
}

• At the end of this file we export both functions so we can use them in our application:

module.exports = {
    createDynamoDBLocal,
    getDynamoDBLocal
};

Accessing a local DynamoDB from Node.js: Using it in your app

Inside index.js or your main app, app.js (the entry point to your application)

If you are an a test or development environment, you can initialize the local database with tables and dummy data, otherwise you can just use the cloud db. We can call this as soon as our server starts.

In my situation, I am using it when running tests, so when my NODE_ENV is set to test. When this is the case, first I will await the tables and dummy data to be inserted.

if (process.env.NODE_ENV == "test") {
    console.log("Creating tables for local DynamoDB docker container")
    const localDb = async () => {
        await createDynamoDBLocal();
    }
    localDb();
}

In your controllers, routes or endpoints, you can refer to the local-db file to grab this local instance and make updates to it directly. Make sure your docker and docker container is running! If you stop and start the container, you will need to re-run the server to run the function that creates the tables.

Add this to the top of the file, above all your endpoints.

const { getDynamoDBLocal} = require('../aws/local-db');
let dynamoDB = '';
if (process.env.NODE_ENV == "test") {        
    dynamoDB = getDynamoDBLocal();
} else {
    // connect to the cloud dynamo db
}

For example, here is a request to delete a record, and we will use the dynamoDB reference we created up above at the top of the file.

const deleteRecord = async (req, res) => {
    try {
        const id = req.params.id;
        
        const params = {
            TableName: process.env.DYNAMODB_TABLENAME,
            Key:{
                "id": id,
            }
        };
        
        const result = await dynamoDB.delete(params).promise();
        res.send({
            success: true,
        });
} catch (error) {
        console.log(error)
        res.status(400).send({
            success: false,
            message: error
        });
    }
}

Hope this helps :)

Good luck!