Originally posted on the DataCamp Engineering Blog.

Last year, we deployed a major re-implementation of the core code editing experience of our interactive courses. In this article, we explore how our engineering team tackled the difficult problem of swapping out one of the major components of the DataCamp experience with minimal disruption to our users, whilst simultaneously reducing technical debt and laying the foundations for future success.

Our journey is broken down into 6 steps:

1. 📍 Our starting point — the “legacy” coding experience and its limitations
2. 🗺️ Choosing our destination — what does the new coding experience look like?
3. 📝 Planning our itinerary — how will we get to our destination?
4. 🛫 Bon voyage — a dive into the journey itself
5. 🛬 Final approach — incrementally delivering our new experience
6. 🏖️ Arrival at our destination — a reflection on where we landed and our next steps

📍 Our starting point

In order to put our journey into context, we need to understand our starting point. One of the core components of the DataCamp learning experience is the code editor, where users can type in their code and see the output of its execution below.

The starting point of our journey is a custom code editor built on top of the Ace Editor library. This editor had served as the reliable backbone of Campus — our interactive course application — for several years. Over time, the limitations of our existing implementation started to show:

• Poor accessibility: We wanted to improve the accessibility of our coding experience. This was difficult to achieve with our current implementation due to the foundations it was built upon. Amongst other problems, it suffered from the “keyboard trap” — the focus of the keyboard would be swallowed by the editor and would essentially be impossible to escape — this was a severe limitation for users who use screen readers and navigate with their keyboards
• Poor abstraction: Due to how it was originally implemented, coupled with “code rot” over the years, the old editor was poorly abstracted and it was difficult to understand where it’s boundaries were within our application. Put more simply, the code was scattered around in different places and difficult to reason about. This made it difficult for us to maintain and build new features for
• A technological dead end: It was built on top of a mature, but “old school” code editing library from an earlier generation of JavaScript libraries, which no longer sees much in the way of active development — limiting future improvements to the user experience
• Bad page performance: Because of blurry component boundaries, it was difficult for us to find a suitable “seam” to split our code bundle. This meant that it was bundled into the main JavaScript bundle of our application irrespective of the type of exercise the user was taking, ultimately slowing down their experience

With these problems in mind, it was time for us to imagine our destination and what that might look like.

🗺️ Choosing our destination

With the problems of our previous implementation in mind, we set about dreaming of what our ideal destination would look like:

• Better accessibility for our users
• A simple abstraction with clear boundaries
• Modern underpinnings with continued development for the foreseeable future
• An overall more simple implementation with less hacks and workarounds that is easier for us to reason about

In most circumstances, we would lean towards doubling-down on an existing implementation and working to improve it slowly over time, an approach that we have seen great success with in the past. However, due to its overall state, in this case we preferred to start from scratch with better technological foundations, but with an emphasis on building and delivering this incrementally to avoid the pitfalls of a “big bang” re-write.

The most important decision we would make is which code editing library to build upon. Building a code editor from scratch is a monumental undertaking and not something we wanted to consider as we certainly wouldn’t do a better job than any of the existing dedicated libraries.

We evaluated several solutions along the way, but ultimately settled on building our editing experience on top of Monaco. Some of you may know Monaco as the sunny microstate host of the Grand Prix on the French Riviera — but it is also the open-source editor that powers Visual Studio Code, a vastly popular and well supported Integrated Development Environment.

Our reasons for choosing Monaco include, but are not limited to:

• Greatly improved accessibility support built-in
• A more modern editing experience for our users, and a more modern API for us to work with, around which we can build a simple abstraction
• It’s supported both by Microsoft and as an open-source project and likely to be developed for the foreseeable future
• It’s modular and allows us to only ship the bits and pieces we need to power the DataCamp experience

With our destination decided, we could now focus our efforts on developing the new editing experience.

📝 Planning our itinerary

Although it did a great job of powering the DataCamp coding experience, our existing code editor implementation was difficult to work with because it was not well encapsulated:

• It was split over several different packages in different repositories. This added cognitive overhead to understanding how it worked and also added friction to the developer experience
• It was also a “leaky abstraction”. Though it was abstracted, there were also little bits of code, CSS and duplication scattered throughout the host application which made it difficult to find a true “seam” or boundary beyond which the module actually lies

These problems made maintenance difficult, and we wanted to give ourselves the flexibility to be able to change the underlying editor in the future without needing to make sweeping changes to our codebase. As a bonus, it would also provide a great leverage point for asynchronously loading the code editor which allows us to defer loading the module until we really need it.

For our new implementation, we settled on the simplest abstraction we could use in this scenario: a React component, that we could drop into our codebase with a handful of props required to interface with the host application:

Being able to integrate our editor into Campus with relatively few lines of code made it easy for it to coexist with the existing editor functionality. As mentioned before, we are aiming to build the new editor incrementally and avoid a “big bang” rewrite/release and having a well-encapsulated component made it pretty trivial to wrap the code path with a feature flag which would allow us to dynamically switch on/off the new functionality to specific users or a certain percentage of our user base (which we’ll discuss more later on!).

With our itinerary planned, we could now embark on our great journey!

🛫 Bon voyage

Now that we know where we’re going and how we’re going to get there, the next step of our journey was to build the new coding experience.

At DataCamp, we write most of our frontend code in React. In order to make Monaco more convenient to work with we used a simple wrapper, react-monaco-editor, which allows Monaco to render more easily as a React component.

We load the editor code bundle itself using monaco-editor-webpack-plugin: this gives us more control over how the editor is bundled and allows us to pick the different elements of Monaco that we need. For instance, select only the language support that we actually use in our courses such as Python or R. By only loading support for select languages and features, we are able to keep the bundle size down which means it will load quicker in our user’s browsers.

Although much of the editor heavy lifting is provided by Monaco itself, we also have a large amount of custom logic that is essential to the DataCamp interactive coding experience:

• Custom keyboard shortcuts and behaviour: for instance, Shift+Return will execute the current line and move the cursor to the next line
• Custom error highlighting: when you submit an exercise on DataCamp, the backend may return a code location which we can then use to render an error
• Custom syntax colouring: the final version of the new editing experience would be shipped as part of our 2020 brand refresh, and so everything needs to look the part
• Custom autocomplete behaviour: the code suggestions that you see while editing code on DataCamp is inferred both by tokenizing the exercise code, but also by executing commands on our backend The removal of various Monaco features that are not required in our editing experience, for instance the built-in Command Palette

We set about implementing the above without falling into the trap of our previous version: we made sure to implement the features as simply as possible and at all times kept them encapsulated within the confines of the interface that we defined above in order to avoid a “leaky abstraction” and a maintenance nightmare in the future.

Unfortunately, it was sometimes necessary for us to use “hacks” or workarounds required to implement some of the custom logic. However, we keep these to an absolute minimum and documented when they were required.

There’s also another aspect of the DataCamp coding experience that we haven’t mentioned yet: the interactive console. The console is where users can type in ad-hoc commands/code and see the output in a scrollable buffer.

But why are we talking about this in the context of the code editor? If you take a closer look at the console component you’ll notice that there’s more to it than meets the eye. In order to create a unified user experience, the text input where the user types needs the same text editing experience as the main code editor — including the same autocompletion experience. You may be surprised to learn that that single line text input is also an instance of the Monaco editor!

Whilst the previous Ace implementation of the console component was also an editor instance, it also included a lot of ugly workarounds involving read-only cursor selections to manage the code output and history. This time around we were keen to keep the complexity to a minimum and so we opted to write a much simpler history buffer and used Monaco only for the actual command line where the user types.

We built our new code editor component incrementally over a couple of sprints (we typically use 2-week sprints) and by utilising the feature flag we were able to deliver the functionality incrementally and iteratively, which we talk more about as we begin our final approach to Monaco!

🛩️ Final approach

With our new implementation in place and safely tucked away behind a simple abstraction, we were nearly ready to go live. However, with such a fundamental under-the-hood change, we were keen to mitigate any major problems.

Much like a plane reducing its altitude on final approach for landing, we opted to release our new editing experience in incremental stages:

1. We first shipped the feature internally to gather feedback from staff
2. We then rolled out to 10% of our user base and increased this by a further 10–20% each week as our confidence in the new implementation increased

Both of these steps were extremely valuable for us to gather feedback and iron out any last-minute quirks. We have several DataCamp employees who are also heavy users of the platform, and some of the most valuable feedback came from them:

With a few course-corrections in place we then began to scale up to our wider user base and corrected any bugs or small changes in behaviour as they were reported.

The final 100% rollout of the new editing experience was coordinated to occur at the same time as our platform-wide brand-refresh of 2020. In October 2020 we finally flipped the switch and the new coding experience was live for our entire user base!

However, there was one final step to take. Due to the incremental rollout we had performed, the main codebase of Campus essentially contained two full code editors side-by-side: the legacy experience, receiving 0% of the traffic, and the new experience receiving 100%. The last step was simply to delete the old editor and all its code paths from our codebase!

🏖️ Landing at our destination

It’s been a long journey, but we’ve finally arrived at our sunny destination! The sea is warm and the piña colada is cool. Let’s take a closer look at exactly where we’ve landed. But first, let’s quickly reflect on where we started:

• A codebase that was difficult to work with and change
• An editing experience based on an older and not actively developed editing library
• Limited support for accessibility features
• A poorly abstracted implementation that had to be loaded synchronously with the main application bundle

In contrast to where we are now:

• A simpler codebase that is well abstracted and easier to maintain
• Built on top of Monaco, a more modern and actively developed editing library
• Greatly improved accessibility features — and no keyboard trap (Ctrl+Shift+M on Mac, Ctrl+Shift on Windows/Linux)!
• As a bonus, can be loaded asynchronously due to a clear abstraction boundary which helps us to manage performance
• A better experience for our users!

🕶️ Eyes on the horizon

Let’s take a look at some of the key learnings from our great journey and look towards the future.

Pick the right abstraction

One of the key problems with our previous implementation was that it was poorly abstracted. With a leaky, but also over-extracted design, it was difficult to find the boundaries of our previous implementation, and subsequently difficult to work with. Aggressively reduce complexity

Aggressively reduce complexity

Due to limitations of the foundations that we built upon, our previous implementation contained many hacks and workarounds, when often there was a simpler approach available. Stand on the shoulders of giants

Stand on the shoulders of giants

Our original editing experience suffered from poor accessibility, and also few prospects for improving the status quo. By building upon newer and better supported foundations we are able to provide a better experience for our users for the foreseeable future. Build and deliver incrementally

Build and deliver incrementally

One of the key tenets of Agile is to build and deliver incrementally, and this project was no exception. Avoiding a “big bang” release allowed us to deliver value incrementally, incorporate feedback from our customers and ultimately mitigate the risk of changing one of the most fundamental aspects of the DataCamp coding experience.

💭 Closing thoughts

Though we find ourselves in a better place to where we started, our journey is not yet over. We have laid the groundwork for success but there is still much work to be done:

• Monaco is an extremely capable platform to build upon: though we get a lot for free, we have not yet begun to reap all of the benefits by way of innovating the core user experience of our editor
• We have improved the accessibility of the code editing experience, but there is still a significant amount of things we could do better and will continue to do so in the future, not just within our courses, but across the whole DataCamp platform
• We are taking a closer look at performance this year, and there’s no avoiding that Monaco is a large library to ship to a browser. This is especially impactful to our customers in developing territories where performance and bandwidth is more scarce. We have only scratched the surface of the improvements we can make to the amount of code that we ship and will be working on this in the coming weeks and months
• Due to the requirement of supporting very old browsers, we are based upon an older version of the Monaco editor. We have recently dropped support for some older browsers which means we can now embrace the latest versions of Monaco.

Overall, we’re happy with how this project played out and learned valuable lessons about swapping out a key part of the DataCamp learning experience with minimal disruption to our learners. We hope you enjoyed the trip!

By the way, DataCamp are hiring! There are a lot of open positions across many departments including Data Science and Engineering. Come and see, we’re eager to meet you!