From Data Scientist to Software Engineer

Hi there! 👋

I’m Paul, Software Engineer at Joko for 2+ years now. In this article, I’ll walk you through my transition from Data Scientist/Machine Learning Researcher to Software Engineer in the company. I’ll describe why I transitioned, how Joko made it possible, and what it brought to me.

👶 Background

Growing up, I really liked studying sciences, particularly math. It naturally set me up to pursue an engineering career, and so after high school, I prepared for the entrance exams to French engineering schools.

I got accepted at École Polytechnique in Paris, France, where I had the opportunity to discover diverse scientific fields at the beginning of my curriculum: not only applied maths, pure maths, and computer sciences, but also physics, economics, biology, civil engineering, and others.

Though I enjoyed the abstraction of pure maths, I loved even more working on real-life concrete applications. So naturally, I chose to specialize further into applied math and computer science.

🔢 Career Choice: Data Science

At the time (around 2016), the Data Science/Artificial Intelligence hype was taking over the world.

In fact, despite École Polytechnique being a very generalist engineering school offering a broad range of studies in various sciences, a third of the students in my graduating class chose to study Data Science.

Data Science was a perfect match for me, as it combines a huge math component and heavy theoretical developments with a lot of algorithmics and hands-on software development, to solve real-world problems.

I further specialized in Data Science/Artificial Intelligence with the MVA research master, which offered in-depth theoretical courses on both fundamental and advanced algorithms, and interesting group projects to apply these algorithms.

These group projects were indeed real-world problems, but at the time I noticed they usually took place in a ”controlled hypotheses” setting: data were already collected and cleaned, small scale and the algorithms to implement had no real speed or scalability constraints (only performance constraints). It was the first time I felt like I was missing “something” to apply Machine Learning to real-world problems.

To complete my master’s, I had to do a research internship within a lab or a company, and that’s when I first met Joko. At the time, Joko had only been created a year prior and was not yet building the all-in-one shopping app and the mobile browser for shopping we are working on today, but a reward app based on open banking, with a quite small team (less than 10 people).

The discussions I had with the team during the interviews really hooked me up to the company and its mission. I was convinced I could have a great impact on concrete problems that millions of people face everyday, unlike in many scale-ups/big companies where I would only be a very small piece of a large machine, or in some labs where research appeared to me as too disconnected from real-world issues.

🧪 Machine Learning Research Internship at Joko

I began my internship in April 2019, as Joko’s first Data Scientist. I was excited by the high ownership given to me to kick start the Machine Learning research practice, and hooked by the very benevolent yet ambitious atmosphere stemming from this small team.

I worked on developing graph algorithms to recommend the most relevant brands to our users within the app, based on their shopping interests.

I faced 2 main problems with most of the well-studied algorithms in the literature (including the “fundamental” ones):

• They did not perfectly apply to Joko’s specific setting
• The scalability aspect was never the top priority, particularly for state-of-the-art algorithms or algorithms that were more suited to Joko

For my research to be actionable and applicable for Joko, I had to focus a lot on the speed/scalability/adaptability aspect, and so I had to dive deeper into implementation and optimization.

In parallel, the Software Engineers were iterating on the product at a fast pace. I could observe a lot being done, in a small amount of time and with a small team.

When I started my internship, these Software Engineering projects seemed more “basic” than the research work, but later appeared to be much more complex than I initially thought and with a much higher impact on the short/medium term than research work.

I learned a lot during this internship, and realized that:

• Building any impactful Machine Learning architecture/product requires a lot of Software Engineering skills and efforts
• As a consequence, in small companies/organizations, Data Scientists also have to be Data/Software Engineers. Machine Learning studies do not prepare for that because they put us in a completely different setting (”controlled hypotheses”)
• Software Engineering is a complex science too, with its own challenges that can be as much interesting as Machine Learning. In particular, building scalable, robust, and clear architectures/code require a different form of abstraction than math, but particularly enjoyable and rewarding
• Software Engineering work is very rewarding: it is the “software” equivalent of being both an architect and a mason. It is also very diverse: there is nothing in common between building a mobile browser and issuing virtual credit cards (but both are topics we are working on today at Joko!)

I wanted to acquire these software skills to be a much more complete Data Scientist/Engineer.

At the end of my internship, I was offered to pursue at Joko as a Software Engineer. At the time, we were still quite small and had a lot to build on the product, so we decided to focus most of our efforts and resources on Product and Software Engineering.

I had the choice between continuing my just-started Machine Learning career or transitioning to Software Engineering at Joko.

You probably guessed it by now: I chose the latter. I wanted to learn Software Engineering to become a much more complete (Machine Learning) Engineer, and I knew Joko was a great place to make this transition, even though I had very limited knowledge of Software Engineering at the time (mostly basic Python skills).

🧳 The Journey From Data Science to Software Engineering

I thus transitioned into a Software Engineer role at Joko for my first job, and since then I learned about software architecture and development, project management, technologies, and even recruitment.

I found Joko to be a great place to learn Software Engineering. We value analytical skills, curiosity, and the desire to solve complex problems, much more than previous experience/knowledge with Software Engineering or specific technologies. As such, we often consider Software Engineering applications from people with no previous Software Engineering experience, for example, Machine Learning Engineers (like myself), or quantum physicists! (check out this very cool article from Alice, Software Engineer at Joko).

We also make it possible and enjoyable for engineers to learn full-stack Software Engineering from scratch through:

• a gamified onboarding process that gets them to learn the basics of our tech stack during their first weeks at Joko, and ship their first production feature during week 1
• the organization of the engineering team in Areas of Focus™️, i.e. “thematic areas” of the product that a single engineer or a duo tackles end-to-end with high autonomy and ownership, allowing us to progressively work on every part of our stack (mobile/web app, browser extension, API or backend)
• committing the time to do extra thoughtful code reviews, putting the emphasis on constructive, actionable feedback that will quickly get them up-to-date with our conventions and coding best practices

As a result, after 2 years I now feel quite experienced with modern full-stack Software Engineering technologies (serverless cloud-computing on AWS, React & React Native apps, TypeScript, and Docker, among others) and practices, and able to tackle increasingly complex projects.

For example, as part of the “products integration & searching” area of focus I’ve been leading since the beginning of 2022, I worked on building a product catalog, tracking the prices of its items over time, and making this catalog accessible to users through an efficient & relevant search engine in the mobile app!

This involved building whole new services from scratch, introducing new types of technologies in the company (e.g. I was the first to use OpenSearch at Joko), and designing complex architectures in full autonomy (e.g. I designed the architecture that allows us to run automated browser instances which embed our product data detection algorithms on the server side).

⏭ 2 Years Later: Not Looking Back

Now, you might be thinking: “it sounds awesome, but don’t you regret not pursuing a career in Machine Learning?”.

Sometimes I do miss the “magic” that working on Machine Learning provides through sorcery-like algorithms achieving impressive AI feats. But Software Engineering brought me another kind of magic: impacting the lives of hundreds of thousands of people through useful apps/services built on elegant, efficient, and robust software, within a small team. On top of that, I feel much better prepared to work on impactful Machine Learning projects again one day.

In reality, that day has already come, as we’ve started to invest extensively in Machine Learning at Joko to build algorithms able to understand webpages just like humans do, to transform the shopping experience of our users (read more about the browser for shopping we build in this article).

Within the Software Engineering team, we now aim to spend around 10 to 20% of our time working on these Machine Learning subjects, assisting the research work (from our research team dedicated 100% to these problems) through participating in brainstormings, high-level planning and prioritization, and through our software architecture and optimization expertise. It means I’m now enjoying both the fun of Software Engineering and Machine Learning!

👋 Conclusion

When I started my Machine Learning studies, I did not expect to later transition into Software Engineering. But thanks to my internship at Joko, I developed a passion for Software Engineering, which I discovered to be very rewarding and enjoyable as it can impact millions of lives, while often providing complex kinds of abstraction.

At Joko, becoming a Software Engineer was made possible and fun for me, through a gamified onboarding, increasingly complex projects impacting our users from day 1, and extra thoughtful feedback from more experienced peers during architecture and code reviews.

Not only am I growing as a Software Engineer, but the skills I am developing are highly relevant to enhance and speed up production Machine Learning projects.

We have very exciting Software Engineering and Machine Learning work ahead, as we aim to build the ultimate shopping app, where we embed on-the-edge Machine Learning algorithms in its browser, to understand, transform and navigate webpages autonomously just like humans do.

If these projects sound exciting to you too, feel free to reach out to us! You can learn more about Joko here. If you want to become a Software Engineer but don’t have any Software Engineering background, don’t worry: we’d be happy to have you learn Software Engineering along the way, tackling increasingly complex projects impacting millions of people!