May 23, 2020
After tweaking a classification model for a few weeks, it came to the point where I needed to share my work with the team for review and collaboration. I had everything setup locally and didn't want to spend alot of time altering the workflow, so I was looking for the lowest friction solution to quickly publish the code. At this point no need to worry about scaling or deployment; ultimately, the project would be managed on AWS but at this stage I didn't want any complex tooling to distract from the actual model building. In terms of basic requirements, the service should have the following:
- A free tier, limited CPU/GPU instances are ok but otherwise fully functional
- Private projects and private sharing
- Package mangement
- Easily upload and work with our own data
Currently, there are a lot of offerings for cloud-hosted notebooks and they all seem promising. Of course, a quick google search will bring you to several reviews and comparison articles that highlight pros/cons of each service - i'll leave a links to a few I found helpful. You may be more interested in performance or what the upgraded accounts offer, but for the time being these weren't a priority.
Among the bunch, a few that caught my eye include: Google Colab, which I often use as a scratchpad, but upgrading is only available in the US; Databricks, the Community Edition is great but no private projects on free tier; Datalore from JetBrains checks a lot of boxes and since I use PyCharm seemed like a great fit; Azure Notebooks, also a contender. All excellent services and each could have accomplished the task at hand, yet each seemed to fall just short in one of the main requirements. After trying a few I came across Deepnote, which is still currently in beta but you can request early access here. So far it seems to have everything I need and nothing I dont (rare these days), which makes it such a pleasure to work with. My basic requirements are fully covered but also has two killer features which really make this environment superb:
- Persistent terminal with Git - this was key, I had already been versioning with Git so why not keep the ball rolling. Azure also offers Github import although I'm not sure how tight the integration is.
- Variable explorer - of course! Wasn't even on my radar but now that it's there why live without it.
A quick look at Deepnote's UI
Working in Deepnote feels as close as possible to working in my own IDE, and merging the project into my existing workflow was nearly seamless. I can import/branch the project for others to make edits and experiment, super easy to upload datasets to the root folder or import from an S3 bucket, package dependencies are auto-detected and can be added manually via terminal. And all the while I can continue to develop locally on the main branch and manage everything with a fairly simple git workflow. They're continually adding new features, notably Teams and Comments were a welcome addition, and a slew of others are on the roadmap. I haven't seen any articles so far on this project but I'm sure it will take off once fully launched, definitely check out the early beta if you can!