In case you work with Jupyter, there's nbdime[0] for better integration with git.
Here is how we do things (in the context of machine learning as a team working together on a project using our platform[1]. This may not be relevant for solo/portfolio projects with few or no expected deliverables):
We use notebooks interactively just to get something we can schedule as a long-running notebook. We don't run training jobs in interactive mode: we schedule the notebook and it's pretty much fire and forget and no hidden state. It runs against a Docker image, and you can close your browser if you want at that point, we have both the computation and the output as it runs. Or you can keep tweaking the notebook, schedule that tweak, and keep on tweaking. We do that to decouple experiments from human time. Users can publish AppBooks, and then try different params using an automatically generated form from the notebooks so we can fiddle with params in a form without opening the notebook. We do that to be able to try things from a form, as in on our phone or allowing a domain expert to train models without the whole notebook experience overhead. This is particularly useful when we use features that the domain expert wants to be able to tweak. We just expose them and let them do their thing.
We do this to track experiments as we automatically detect models, parameters, and metrics without having to write tracking/saving code. All this allows us to have ready models for deployment, which we can do easily. We do that to allow people to deploy their own models instead of asking someone, and be able to use, and let others use the models simply by sending HTTP requests or using a form, instead of bothering with models and their dependencies.
For the collaboration aspect, we use near real-time collaboration to help someone troubleshoot or go over code with someone, and then commit changes. We automatically save checkpoints so people can go back and forth.
We mainly use Python and we built around Jupyter. We took a look at other notebooks but, in our experience, stylesheets aren't what held our machine learning projects back in the past and we decided to keep using Jupyter adding functionality as "natively" as possible. More embarrassingly, we didn't come from a front-end background but from an ML background, and built this around our workflow to enable us to execute projects.
Here is how we do things (in the context of machine learning as a team working together on a project using our platform[1]. This may not be relevant for solo/portfolio projects with few or no expected deliverables):
We use notebooks interactively just to get something we can schedule as a long-running notebook. We don't run training jobs in interactive mode: we schedule the notebook and it's pretty much fire and forget and no hidden state. It runs against a Docker image, and you can close your browser if you want at that point, we have both the computation and the output as it runs. Or you can keep tweaking the notebook, schedule that tweak, and keep on tweaking. We do that to decouple experiments from human time. Users can publish AppBooks, and then try different params using an automatically generated form from the notebooks so we can fiddle with params in a form without opening the notebook. We do that to be able to try things from a form, as in on our phone or allowing a domain expert to train models without the whole notebook experience overhead. This is particularly useful when we use features that the domain expert wants to be able to tweak. We just expose them and let them do their thing.
We do this to track experiments as we automatically detect models, parameters, and metrics without having to write tracking/saving code. All this allows us to have ready models for deployment, which we can do easily. We do that to allow people to deploy their own models instead of asking someone, and be able to use, and let others use the models simply by sending HTTP requests or using a form, instead of bothering with models and their dependencies.
For the collaboration aspect, we use near real-time collaboration to help someone troubleshoot or go over code with someone, and then commit changes. We automatically save checkpoints so people can go back and forth.
We mainly use Python and we built around Jupyter. We took a look at other notebooks but, in our experience, stylesheets aren't what held our machine learning projects back in the past and we decided to keep using Jupyter adding functionality as "natively" as possible. More embarrassingly, we didn't come from a front-end background but from an ML background, and built this around our workflow to enable us to execute projects.
- [0]: https://github.com/jupyter/nbdime
- [1]: https://iko.ai