Armin's blog

Production ready AWS ECS (EC2 Container Service)

Wed, 24 May 2017 00:00:00 +0000

Original post can be found at Xebia

Are you looking to run Docker containers on AWS? Or are you looking to use AWS ECS (EC2 Container Service)? Does it need to be production ready?

If so, then speed up your process and be ready today by looking at my GitHub repository on ECS. It contains infrastructure as code with Terraform for a quick production ready setup. But most importantly it contains information on how the infrastructure is setup. It explains why things are done in a certain way and where to watch for. And finally, shows how to do simple but fully automatic deployments.

If you live in the Netherlands then join me at the Open Kitchen: Simplify DevOps with AWS ECS

Docker containers vulnerability scan with Clair

Fri, 05 May 2017 00:00:00 +0000

Original post can be found at Xebia

When you work with containers (Docker) you are not only packaging your application but also part of the OS. Therefore it is crucial to know what kind of libraries might be vulnerable in you container. One way to find this information is to use and look at the Docker Hub or Quay.io security scan. The problem whit these scans is that they are only showing you the information but are not part of your CI/CD that actually blocks your container when it contains vulnerabilities.

What you want is:

Build and test your application
Build the container
Test the container for vulnerabilities
Check the vulnerabilities against allowed ones, if everything is allowed pass, otherwise fail

This straight forward process is not that easy to achieve when using the services like Docker Hub or Quay.io. This is because they work asynchronously which makes it harder to do straight forward CI/CD pipeline.

##Clair to the rescue

CoreOS has created an awesome container scan tool called “clair”. Clair is also used by Quay.io. What clair does not have is a simple tool that scans your image and compares the vulnerabilities against a whitelist to see if they are approved or not.

This is where clair-scanner comes in to place. The clair-scanner does the following:

Scans an image against Clair server
Compares the vulnerabilities against a whitelist
Tells you if there are vulnerabilities that are not in the whitelist and fails
If everything is fine it completes correctly

##clair-scanner

See here the full documentation about the clair-scanner.

Here is a simpel example on how to scan a container and verify if it has vulnerabilities:

docker run -p 5432:5432 -d --name db arminc/clair-db:2017-05-05
docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.0-rc.0

Now scan a container, that has a whitelisted CVE:

clair-scanner nginx:1.11.6-alpine example-nginx.yaml http://YOUR_LOCAL_IP:6060 YOUR_LOCAL_IP

Docker container secrets on AWS ECS

Wed, 22 Mar 2017 00:00:00 +0000

Original post can be found at Xebia

Almost every application needs some kind of a secret or secrets to do it’s work. There are all kind of ways to provide this to the containers but it all comes down to the following five:

Save the secrets inside the image
Provide the secrets trough ENV variables
Provide the secrets trough volume mounts
Use a secrets encryption file
Use a secrets store

For a complete description and an explanation please see the following post

Depending on your context all five options might work or are considered good enough. In my opinion to do it right and be safe, flexibel and in control you can only go for option 5 “Secrets store”. Choosing this method it leaves you with different ways to accomplish this. As stated in the post above it comes down to this:

Use some form of S3, with or without encryption. For more info see
HashiCorp Vault

AWS has released a service called Parameter Store, although it is not as sophisticated as Vault it serves a purpose. If you are starting or want to keep things simpel please try using the Parameter Store. But if you need full control and want more flexibility then go ahead and use Vault.

###Parameter Store

Parameter Store allows you to store key value data in a key value store with the values encrypted using KMS. Because Parameter Store uses IAM it is a great fit for AWS ECS because you can control the access to the secrets per container.

I could explain how to do this but luckily AWS has a great post on how this should be done, you can find it here

The blog post is great but I am not really impressed with the run.sh script in which I am replacing the secrets and fetching them with aws cli. There has to be a better and cleaner way….

I really thought so but we could not find any, so we chose to create one.

###Dockerfy

Without trying to invent the wheel again we took the simpel but useful app “dockerfy” and extended it. More information on Dockerfy can be found here. Dockerfy already handles ENV’s and secret files, and it uses templating for simpel usage of secrets.

Sadly it did not have a AWS Parameter Store integration but we took care of that, as you can see in this PR

You can use the Parameter Store and combine it with Dockerfy features to add your secrets using a template but if you need a simpel command-line secret, here is an example.

Dockerfile:

FROM socialcode/nginx-with-dockerfy
ENTRYPOINT [ "dockerfy", "--aws-secret-prefix", "PROD_" -- ]
CMD [ "/bin/echo", "" ]

To use the above Docker container, store a password in the Parameter Store called “PROD_db_password”. Then provide the AWS keys or run it on an EC2 instance or an ECS service with proper IAM rights and see the secret being echoed.

Dockerize your Development Workstation

Sun, 14 Feb 2016 00:00:00 +0000

This is my Dutch Docker Day talk “Dockerize your Development Workstation”

GTD (Getting Things Done)

Mon, 28 Sep 2015 00:00:00 +0000

Original post can be found at Xebia

As a consultant I am doing a lot of things, so to keep up I have always used some form of a TODO list. The reason why I did this is because it helped me break down my tasks in to smaller ones and keep focusing, but also because I kept remembering the quote I once heard “smart people write things down, dumb people try to remember it”.

Years ago I read the books “Seven habits of highly effective people” and “Switch”, in my research in to how to become more effective I came in to contact with GTD and decided to try it out. In this post I want to show people who have heard about GTD how I use it and how it helps me.

For those who don’t know GTD or haven’t heard about the two books I mentioned please follow the links Getting things done Seven habits of highly effective people Switch and have fun.

How do I use GTD?

Because of my experience before GTD I knew I needed a digital list that I could access from my phone, laptop or tablet. It was also crucial for me to be able to have multiple lists and reminders. Because of these requirements I settled on using Todoist after having evaluated others as well.

I have couple of main groups like Personal, Work, Read-me, Someday & Maybe, Home etc.. In those lists I have topics like Xebia, CustomerX, CustomerY etc.. And in those topics I have the actual tasks or lists with tasks that I need to do.

I write everything in to my Inbox list the moment a task or idea pops up in to my mind. I do this because most of the time I don’t have the time to figure out directly what I want to do with it so I just put it in to my Inbox and deal with it later.

As a consultant I have to deal with external lists like team scrum boards. I don’t want to depend on them so I write down my own tasks with references to them. In those tasks I write the things that I need to do today, tomorrow or longer if it is relevant in making decisions on the long run.

Every day I finish my work I quickly look at my task list and decide on priorities for tomorrow, then every morning I evaluate my decisions based on the input of that day.

As a programmer I like to break down a programming task in to smaller solutions so therefore sometimes I use my list as a reference or micromanagement of tasks that take couple of hours.

To have a better overview of things in my list I also use tags like tablet, PC, home, tel so it helps me pick up tasks based on context I am in on that moment.

Besides deciding on priorities every day I also do a week review where I decide on weekly priorities and add or remove tasks. I also specify reminders on tasks, I do this day based but if it is really necessary I use time based.

Because I want to have one Inbox I am sticking to zero mail in my mail Inboxes. This means every time I check my mail I read it and then either delete it or archive it and if needed make a task to do something.

What does GTD do for me?

Pros:

The biggest win for me is that it helps me clear my mind because I put everything in there that I need to do, this way my mind does not have to remember all those things. It is also a relaxing feeling knowing that everything you need to do is registered and you will not forget to do it.

It gives me structure and it allows me to make better priorities and decisions about saying yes or no to something because everything is in one place.

Having a task registered so that you can cross it off when it’s done gives a nice fulfilling feeling.

Cons:

Having a big list of things that you think you need to do can be quite overwhelming and in some cases it can also feel like a pressure because you haven’t done a list of tasks that is there for a long time. That’s why I keep evaluating tasks and remove things from it after some time.

Task lists and calendars can sometimes collide so it is importent to keep your agenda as a part of your list although it’s not a list.

What am I still missing?

Right now from the GTD’s point of view I mis a better time management. Besides that I would like to start a logboek of relevant things that happend or people I spoke to but also incorporate that with my GTD lists (for example combining Evernote/OneNote with my Todoist lists).

Everything else I mis is technical:

I mis a proper integration of all my mail where all the calendar invites are working without having to connect all accounts on all different devices. I also mis a proper integration between my mail and Todoist for creating and referencing mail.

Because I write down every task/idea that pops up in to my mind it would be great if voice recognition would work a little better for ‘me’ when I am in a car.

Conclusion

GTD has helped me structure my tasks and gave me more controle in to making decision around them. By writing this post I am hoping to trigger somebody else to look in to GTD and maybe have an impact on his or hers effectiveness.

I am also curious in what your opinion is on GTD or if you have any tips for me regarding GTD?

Rooting my Galaxy S5

Sun, 21 Jun 2015 00:00:00 +0000

It’s been a while I have tinkered with my Android phone, I thought it’s time to root and play around. How hard can it be right? I have done this 100 times before. You already know where this is going, 2+ hours of flashing rebooting until it worked…. :)

There is plenty information about what to do compared to years ago. Mostly if you follow the tutorials it should go right but sometimes it just doesn’t. Let me walk you trough what I did to root my phone and eventually install a custom rom :)

Download ODIN, there are two version old and new one, the ui is different. Download custom recovery, find the new version on xda forums. Let’s start

I used the latest odin, connected my phone it starts flashing but nothing else happens it just hangs showing progress bar. Hum lets try and reboot the phone in download mode, pfff it works. Seems odin does not start as system, lets try again, it works and the phone reboots, I reboot in recovery mode but I end up in samsung recovery, wtf. Ooh it seems there is a script that brings back recovery mode, to fix it remove battery when the flashing is done, before the phone boots and then boot in recovery mode to override the script. Watch out not to remove the battery to soon :)

Custom recovery is there victory :) , make a backup, shit can’t need 8 gig. Lets just root, seems to work, reboot phone, hum stuck on boot screen. Let’s wipe everything, reboot and the phone is still stuck on boot, boot again hum now I see it’s optimising apps… After some time I am in my android but root is broken, damn it. Root again and now the phone is stuck on boot before samsung logo, damn it.

Now I can’t get the phone to boot in any way, I have two options flashing original with odin, but need to wait 2 hours for a download or install a custom rom. Let’s do custom rom, but how do I install it I can’t copy the zip on to it. Oke no problem let’s sidelode it trough adb. It seem to work, phone is rebooting, damn stuck at boot again. Reboot to recovery again and wipe everything, try again, seems to work phone reboots, stuck on boot again, I wait a minute or 2 and reboot, ooh I see it’s optimising apps. It seems it was working on the first boot as well only it was optimising apps but not showing it. Now I am in a running android with root working whooooo. Finally I can use my phone again.

Now only thing I need to do is install some of the samsung apps trough recovery to allow exchange support etc.. But also install all my apps and configure them, have no backup of them… Luckily there is lastpass helping with passwords :)

No mater how many times you have done this it’s alway journey, I really need to fix/root my phone when I buy while I still have my other phone working so I have more time to mess around

Continuous Delivery of Docker Images

Fri, 12 Jun 2015 00:00:00 +0000

This is my Xebicon talk “Continuous Delivery of Docker Images”, here are the slides

Start again with jekkly

Sun, 07 Jun 2015 00:00:00 +0000

I have not blogged for quite some time now, I just had no fun with it anymore. But as I have done very cool things in the past two years I am actually sad that I haven’t shared my knowledge with the world. I still believe sharing is a way forward for all of us as this is how I learn on the internet as well. I have switched my blog from wordpress to jeklly hosted on github because it is fast and easy compared to the previous blog. I hope this will help me start blogging more even if they are quick posts.

Goodbye Ubuntu, Welcome Arch Linux

Sun, 17 Nov 2013 00:00:00 +0000

I started using Linux years ago and I tried a lot of distributions but Ubuntu was always my main distribution on my work laptop. The reason why I used Ubuntu was simple, it worked and it did not have a lot of maintenance if you didn’t customize. But after so many years of Ubuntu I needed a change.

As I said before I tried a lot of other distribution before but always came back to Ubuntu. I think the reason for this is that I never really prepared myself for the switch but I just did it. So this time I wanted to do it differently. First I made a short list of the distribution that I would like to use, this was a quick one.

Arch, because it becomes my system and I decide what happens with it and because it is bleeding edge
Debian, rock solid
Red Hat Enterprise, I needed a third option

I didn’t really wanted Red Hat and if I am going to switch from Ubuntu to some other distro I needed a bit more change than there is between Ubuntu and Debian (and no I am not saying they are the same!). I have looked in to Arch before and even installed it on my laptop but didn’t give it a change, my main reason to install Arch is to learn more about what my system is doing and to be on the edge of software. But instead of diving in to it as I normally would I went and prepared myself.

I read trough the installation, I looked at different options, made my choices regarding boot manager,artitioning, file system etc… Then I installed it in VirtualBox and started configuring and testing the software I thought I wanted to use, this way I tried new software and decided on my defaults. After that I went and installed Arch on my laptop and since 4 weeks Arch is my main distribution that I love!

Parallelism simplified by Akka in Java

Fri, 30 Aug 2013 00:00:00 +0000

I have bean writing applications in Java EE environment for some time now and I didn’t need to worry about creating own threads. When I needed to create and manage my own threads it always felt wrong because of to much low level programming. Then I came across akka and wanted to try it out.

What is Akka?
Quote from akka.io “We believe that writing correct concurrent, fault-tolerant and scalable applications is too hard. Most of the time it’s because we are using the wrong tools and the wrong level of abstraction. Akka is here to change that. Using the Actor Model we raise the abstraction level and provide a better platform to build correct, concurrent, and scalable applications. For fault-tolerance we adopt the “Let it crash”; model which the telecom industry has used with great success to build applications that self-heal and systems that never stop. Actors also provide the abstraction for transparent distribution and the basis for truly scalable and fault-tolerant applications.”

Context
For the purpose of the sample we have a trivial working unit that simulates CPU intensive task that can find a factorial for a large number. We assume that we need to find factorials for many different numbers, for the purpose of consistency the number is now fixed.
Factorial calculator

public class CalculateFactorial {
    public BigInteger calculate() {
        BigInteger fact = BigInteger.valueOf(1);
        for (int i = 1; i <= 739; i++)
            fact = fact.multiply(BigInteger.valueOf(i));
        return fact;}}

Java way
The most simple way of implementing this is to loop through the numbers and calculate the factorial for them, like this.

public class JavaWay {
    private final long messages = 100;
    
    public static void main(String[] array) {
        new JavaWay().run();}

    private void run() {
        time.start();
        calculateFactorial();
        time.end();
        printElapsedTime(time);}

    private void calculateFactorial() {
        for (int i = 0; i < messages; i++) {
            list.add(new CalculateFactorial().calculate());}}

    private void printElapsedTime(Time time) {
        System.out.println("Done: " + time.elapsedTimeMilliseconds());}}

This is a simple solution that works well, but the problem with it is that the processing speed entirely depends on one thread on the local machine even though the messages could be processed simultaneously. It is possible to solve this problem in many different ways, for example by using a container managed environment. Although it can solve the problem it brings complexity and overhead to the project. Another way is to use threading, but that means working on a low level coding and it does not allow distributed scaling. But there is also Akka way.

Akka way
Akka can be used as a library like in this sample, as a microkernel or part of the Typesafe Platform. This sample consists of four parts: bootstrapping Akka (starting ActorSystem), master actor that handles the messages, worker actor that handles the calculating process and at last there are triggers (messages).

Bootstrapping Akka consists of starting the ActorSystem, creating the master actor and telling it to start processing.

public class AkkaWay {

    public static void main(String[] args) {
        new AkkaWay().run();}

    private void run() {
        ActorSystem system = ActorSystem.create("CalcSystem");
        ActorRef master = system.actorOf(Master.createMaster(), "master");
        master.tell(new Calculate(), ActorRef.noSender());}}

Master actor is responsible for processing the messages, meaning telling the Worker actor to calculate the factorial. Akka has different ways to scale, in this sample RoundRobinRouter is being used so that multiple Worker actors can do their work at once. Master worker is also responsible for receiving the results from the Worker actors and knowing when all messages are processed.

public class Master extends UntypedActor {

    private long messages = 100;
    private long processed = 0;
    private ActorRef workerRouter;
    private final Time time = new Time();
    private ArrayList list = new ArrayList();

    public Master() {
        workerRouter = this.getContext().actorOf(Worker.createWorker().withRouter(new RoundRobinRouter(4)), "workerRouter");
    }

    @Override
    public void onReceive(Object message) {
        if (message instanceof Calculate) {
            time.start();
            processMessages();
        } else if (message instanceof Result) {
            list.add(((Result) message).getFactorial());
            if (list.size() == messages)
                end();
        } else {
            unhandled(message);
        }
    }

    private void processMessages() {
        for (int i = 0; i < messages; i++) {
            workerRouter.tell(new Work(), getSelf());}}

    private void end() {
        time.end();
        System.out.println("Done: " + time.elapsedTimeMilliseconds());
        getContext().system().shutdown();}

    public static Props createMaster() {
        return Props.create(Master.class, new ArraySeq<Object>(0));}}

Worker actor is responsible for the actual factorial calculation and returning a response to the parent (master actor) when done.

public class Worker extends UntypedActor {

    @Override
    public void onReceive(Object message) {
        if (message instanceof Work) {
            BigInteger bigInt = new CalculateFactorial().calculate();
            getSender().tell(new Result(bigInt), getSelf());
        } else
            unhandled(message);
    }

    public static Props createWorker() {
        return Props.create(Worker.class, new ArraySeq<Object>(0));}}

For communication between actors Akka uses triggers (messages) that are immutable objects.

public class Calculate {}

public class Result {

    private BigInteger bigInt;

    public Result(BigInteger bigInt) {
        this.bigInt = bigInt;}

    public BigInteger getFactorial() {
        return this.bigInt;}}

This is all the code that is needed to take advantage of parallelism. If local scaling is not enough Akka also has a simple distributed solution build in that allows the Worker actor to be run remotely without changing the code.

Speed: Java way vs Akka way

Messages	Java (milliseconds)	Akka quad (milliseconds)
100	53	38
1.000	302	139
10.000	2661	1072
1.000.000	247579	102315

Conclusion
This sample shows that it is quite easy to take advantage of parallelism by using Akka to speed up the application. Because Akka is on a higher abstraction level than threads themselves it is possible to speed up the application even further by changing the actors number or by running it distributed.

Sources
The working project and the complete code can be found on github.

For more information there are books like Akka concurrency and Akka in action or watch a talk from Jonas Bonér