Dos and don’ts for new projects

Start with the right foot and don’t waste time

When starting a new project it is important to set things right from the beginning, so that you will not waste time later and will not be forced into something that is not the right direction. Here’s my personal list of thins I do and I don’t (mostly based on AWS because that’s what I know more, but any cloud provider is fine).

• Try to offload as much as possible and focus on your core features. You should not waste any single minute doing non-core things, life is too short to reinvent the wheel over and over. The list of non-core things is huge: email, notifications, authentication, payments, infrastructure, and so on. Also, people that spend their life focusing on a single problem have had more time to think about it, know some subtle things you could not, and, in general, have more firepower to tackle the problem.
• Use Docker and anything that manages Docker as a service. Using Docker allows you to easily move your application through different stages (dev, QA, production). It also makes it easier for new hires to set up the development environment and start contributing to the project. I’ve tried Amazon ECS and I have mixed feelings, but it does its job.
• Use anything that keeps a database up and running. Managing a database is a living hell. It is difficult, tedious, and risky (nothing is annoying like filling up a database hard disk, and nothing is scaring like checking the database backup in the case of a disaster; you want both of these features managed by someone else). There is not a single good point about it that comes to my mind. I’ve had a great experience with Amazon RDS and I can recommend it hands down.
• Put CI/CD pipelines in place before anything else. Pipelines are your best friend. Nowadays it’s easy to build a pipeline: they are somehow standardised (wrap your steps inside a Docker flow), and most of the git repositories have integrated which also provide a free tier. There is no excuse to postpone this task; do this first thing and it will save tons of time later.
• The same goes for the testing framework. This is more complex than setting up pipelines; testing frameworks are less standard since they depend both on the tech stack you are using and on how you are using it. Nonetheless, you should write your tests before your code. I know that this is not always realistic, sometimes you just need to release something, or simply you are too tired/bored to write tests before. However, you must have a working testing environment before you start writing code; also add a few examples on how to test your code (not code in general, but exactly the code you have in your repo). The idea here is to do that extra mile that goes from the testing framework tutorial to have it running in your case so that anyone can easily add the tests you didn’t write upfront when you were in hurry. These two things might slow your first commit but will speed up starting from the second.
• Find services that allow you to delegate whole chunks of application. For example, don’t config an email service, use SendGrid. Remember to wrap it, so that it is easy to change it later without modifying every place where it is used.
• Resist fancy tech unless you really need it. You probably don’t need NoSQL, K8S, Kafka, and so on. They are powerful tools, designed for complex scenarios that might not fit your needs, at least at the beginning of a project. Also, they are difficult to use, integrate and maintain, so they will pave the road for future problems …
• ... but use any technology you know. If you are good with K8S use it!
• Don’t do your own authentication system. OK, I’m biased (I’m building an open-source authentication system ), but it’s really a bad idea to do it yourself. If you don’t like my project there are many others (Auth0, AWS Cognito, SuperTokens, KeyCloak). If you can use something open source, it’s better because you keep some control over your data (but, again, I’m biased here).
• Try to avoid vendor lock-ins. It’s a difficult balance between execution velocity and vendor dependency. You should use any service that allows you to easily move away from it, and carefully evaluate services that keep you there. For instance, RDS is a no-brainer here because at its base it’s just a standard SQL server, so you can dump your data and move them. AWS Lambda is a much more difficult choice because it’s difficult to move to another provider once you have tens of lambdas deeply integrated with AWS API Gateway. Auth0 is another different level since it is somehow standard (all in all it provides you a JWT), but it keeps your data and it’s painless to migrate them. If you can, plan migration procedure for services, but don’t waste too much development time. In the end, it’s better to have a locked-in something that a completely free nothing (just in software development, not in life!)
• I find it useful to write my backend in node, so that it is easy to do both frontend and backend, but many people don’t like this. Whatever you do, try to keep your stack simple, and make sure that anyone is ready to be up and running and contribute in a couple of hours top. Docker is your friend here too.
• I’d suggest skiping microservices until you know more about your project. When you go for microservices keep in mind that it involves a lot of infrastructure (sound CI/CD, automatic deployment, and distributed logging at least). Solve infrastructure before going for microservices. Don’t ever dare to do microservices if your pipelines are not good enough (personal experience speaking here). This comment is 5 years old but still applies.
• repl.it has helped me many times to solve small algorithm issues without having the complexity of the project. If your stack is JS, there is an extension for VSC called Quokka that might help.
• Follow the 12factor, if it applies. Smart people carefully thought about it and it is full of wisdom. Think about production early in your development stage, even if you are not releasing anything yet. Production constraints might change something deep inside your code that will have a large impact. For instance, if your project will need to handle multi-tenants in production, it is better to have a multi-tenant-capable database code before writing tens of services that will interact with the database. Also, if you need to scale services on multiple servers, your code shouldn’t be much stateful. The 12factor helps you to avoid some pitfalls (like not fetching configuration from environment variables), but you should think also beyond this.
• When in doubt, wrap it. If you are not sure you can use a service, or follow a given approach, if possible, wrap it into a function. Later on, you can re-implement the function without changing everything else. I know this is a best practice and everyone knows it, but it’s good to remind it.

I’ve learned each one of these points by hitting walls and earning sleepless nights. I hope that by sharing this I can help others to avoid some troubles and to release faster and better.