Protecting Redis with SSH Tunnels in Multi Cloud Environment
Okay! I know the title is too verbose,but I found it attractive.
These days I m working on a Project call GStatic , It’s a static site hosting service for developers.
Let me come to point, now.
For GStatic we are using redis as a messaging bus so that are micro services can communicate to each other. We are using Redis Pub/Sub to push orders to some of our working nodes.
Our infrastructure is hosting on AWS and Digitalocean for now, as they are providing a lot of free services. As Redis is not that protected, we decided to make it public just for testing purposes and making things simple, Everything was going smooth we were about to launch the MVP. But after deploying it for 45 secs all the services which were polling redis got crashed, and I was like “What the heck”. After googling for half an hour, I got an open Git Issue which said that it can be an attack, Redis allow its client to change its configuartion using CONFIG command. I was like, “Dude, What are saying ?”. But it was true we got hacked.
There was to two solutions, either Password Protect redis or Put all the services at one place i.e. in same VPC. First Solution requires changing code on 4 services, which is pretty boring and frustrating task. Second approach was not economic for us, as we can’t pay for 2 EC2 instances. I was scratching my but nothing strike me.
Solution
Next morning, I was thinking about on my previous project Shree, an open source Tool to make SSH Secured Port Forwarding Tool, pretty much like ngrok. It got me an Idea What if using SSH Tunnel to forward my redis port(on EC2) to my digitalocean droplet. So did the same, It took me only one command
Let’s deconstruct above command,
ssh : The linux command which implements Open SSH Protocol.
- n : Disables Standard In, makes Tunnel connection more reliable.
- -N : Opens ssh connections only for Port Forwarding.
- -l : It tells the username on the remote machine to connect with
- -i : Specifies identity file, which is nothing but RSA Private key of the server to be connected via ssh.
- -R: This specify that Port forwarding will be Remote.
- 6379:127.0.0.1:6379 : First port 6379, tells that remote host will listen on this port 6379, 127.0.0.1 will the the bind address for the socket which remote host will listen on, last 6379 port says that port 6379 on local host will be forwarded.
- mydroplet.domain.com: This the domain name of the remote machine to connect with.
This one line command saved us, couple of bucks and a lot of frustration. It was working just fine. But SSH have a surprise for us,
Above command only allows connection on loop back interface i.e. to get connected with redis on remote machine (in our case droplet), We can only dial 127.0.0.1:6379 or localhost:6379. Reason?? Because in above command we bind the 127.0.0.1 address on remote machine. I have fix for it
What we did, is we bind the socket on remote machine at address 0.0.0.0 now it can listen on look back interface as well as ether net interface, and That’s how our docker containers connect with redis. And it More secure than Password Authentication provided by redis.
