Emulate AWS SQS offline

Simple Queuing Service (SQS)  from AWS is the first choice for many developers when it comes to asynchronous processing. Entirely managed by AWS with easy to understand documentation and a plethora of resources available online, SQS is a fan favorite. However, while learning or experimenting with SQS, you don’t always want to connect to the actual service because obviously, it incurs some cost. Here is a way of emulating AWS SQS offline in a docker container, connecting to it in a similar was you connect to the actual SQS (using CLI or AWS language-specific SDK) and start experimenting. Please note that this article will use golang as the programming language on Arch Linux.

Recently at my work, I had a situation where I couldn’t connect to  AWS and had to test my code with SQS. After loads of research, I came across a docker image alpine-sqs . It provides a containerized Java implementation of the Amazon Simple Queue Service (AWS-SQS). It is based on ElasticMQ running Alpine Linux and the Oracle Java 8 Server-JRE. It is compatible with AWS’s API, CLI as well as the Amazon  SDK. This allows for quicker local development without having to incur infrastructure costs.

Setting up the container:

Said that, if you are just here to experiment, you don’t actually are very curious as to what is running inside the container. So let us see how to get it up and running.

• • • • • Install docker.

          pacman -S docker

        • Start the docker service.

          systemctl service docker.start

        • Now that we have docker installed, let us take a pull of the image and get it running.

          docker pull roribio16/alpine-sqs
          docker run --name alpine-sqs -p 9324:9324 -p 9325:9325 -d roribio16/alpine-sqs:latest
          

          I have mapped the ports to 9324 and 9325, you can choose any.

        • To check if everything went well, just check use the docker ps command and check if the apline-sqs container is running.
        • Now let us try to install AWS CLI.

          pacman -Sy aws-cli

        • Setup AWS CLI to connect to the AWS server (here the container).

          aws configure          
          AWS Access Key ID [None]: notValidKey
          AWS Secret Access Key [None]: notValidSecret
          Default region name [None]: eu-central-1
          Default output format [None]: json
          

          This is an interactive command, just enter AWS configure and all the options pop up. If you are more curious about these credentials, look here

        • Let us now try sending a message to SQS.

           
          aws --endpoint-url http://localhost:9324 sqs send-message --queue-url http://localhost:9324/queue/default --message-body "Hello, Scribber!"
          

          This command returns you msgID unique to every message sent.

        • This was producing the message, let us try consuming it. The container already provides a dummy endpoint to see the messages sent to the queue. Just point your browser to localhost:9325.

        And there it is, you are good to go and experiment with SQS.

        Connecting with Golang SDK

        I found connecting to Alpine-SQS via Goland SDK a bit difficult, as it was not mentioned in the doc to pass custom URL, so as a bonus I am also leaving here a Golang script to connect to Alpine-SQS.

         
        func main() {
        sess, err := session.NewSession(&aws.Config{
                Region: aws.String("eu-central-1")},
            )
        svc := sqs.New(sess, aws.NewConfig().WithEndpoint("localhost:9324"))
        }
        

        And that is it, you can now use all the functions described in the official AWS Golang SDK and it will be routed to the container. To go live just remove WithEndpoint(“localhost:9324”).