Using this and Gitlab and Netlify you can make static pages that can edit themselves without having a backend anywhere. That is the holy grail of dynamic static. Or Naught Static Pages as I call them :)
I'm not talking about making use of some "serverless" backend like Firebase or something. I'm talking about only using git hosting and static website hosting and in browser computation to make "dynamic" websites.
What you do is:
* You make a regular static page with whatever static web page framework or view template software you like that runs in the browser. I got Handlebars to work. (I'm looking at trying Choo next, because it is literally one hundredth of the size)
* Make another "admin" page where the user can input the data. This page first checks out the current git repo.
* When the user hits submit you use the newly edited data to build the static site in the browser and commit/push everything to git where it is automatically pushed to netlify.
You first have to bootstrap this process on your own machine in the first commit.
I've been holding on to this secret for over a year. Well the cats out of the bag now. I haven't run into someone doing something similar with Isomorphic-git yet. I've had it in production for an artist website for almost a year. I don't know if this is good or bad for Gitlabs or Netlifys business model.
It looks like Netlify-cms runs the git and static website software on Netlify backends. So you're not getting the freedom to do whatever you want in the frontend like you do with my approach where the static site generation and git handling happens in the browser. With Netlify-cms you are tied to their implementation of these things on their backend servers.
You can configure netlify-cms to go through netlify for the git operations or not. Going through netlify servers let users edit without github credentials.
On the blog where I have set this up, it goes direct and content editors aren't that happy with the experience
Doing static site generation in the browser would definitely help with previews, at the cost of not working with off-the-shelf static site generators.
I like the "dynamic but static" idea ! Doesn't this already exists if you have a static website but with some dynamic features (js client code calling some serverless functions or querying stuff on AWS for example) ?
I guess here you are adding the "you can edit it from a browser", is that right ?
I was about to make a site just like you describe, having a static website that uses js to dynamically show and edit some content and have it be fetched and stored at some serverless host like Firebase. I guess AWS has something similar to Firebase, but the complexity of AWS keeps me away from checking it out.
There are two benefits to my approach, which motivated me to do it.
1. No backend code or database. Not even serverless. All the complexity of managing this is gone. Don't have to worry about quotas or paying for this either as it doesn't exist. We just need a static site host that can pull and host from git. Netlify does this.
2. Perfect caching performance, as we build a new static website when we change any data. Nothing is generated at view time. The thought of each user calling a backend for the same data over and over on each page load, when the page hasn't changed gives me pain. With this all the data is just built directly into the static site and the static site host and the browser takes care of doing http caching if it isn't changed.
Dynamic page content without hosting a backend. Backend hosting can be enormously complicated when you just want to be able to edit a few fields in the page dynamically.
My experience is that the "backend" of a website is where the backend business logic code and the database of the web page is. But in this case it is all running in the frontend in the browser.
I'm not sure this is a secret, I think I've seen people talk about this in the past. I even did a little bit of work looking into whether or not it was feasible as a replacement for CMS backends like Wordpress, but eventually gave up because:
A) Git operations on both Github and Gitlab are kind of slow (compared to normal AJAX requests).
B) It requires you to embed Git credentials into your webapp clientside, and I'm more comfortable building secure client sessions than I am messing around with libsodium to encrypt credentials locally.
C) The lack of isolation between my data and site code means that I don't have defense in depth if the credentials get compromised. If someone breaks into my session, they can't only compromise my database, they also have push rights to my entire repository. This is especially problematic for apps where I want multiple people to have access, since I often don't trust them to keep their passwords secure.
D) To expand on that, you can't do any validation of input at all -- clientside validation doesn't count. Your only permission you can grant people is "100% commit access to my entire repo, plus (potentially) the ability to execute arbitrary code during my build process", or "nothing".
E) It's also just honestly a lot more complicated than normal hosting. There ended up being a lot of niggly details that I was fiddling around with that weren't terrible to solve, but left me asking, "what exactly is my end-goal here and is it worth the time I'm putting in?"
The security issues mean that having an open submission form is a pretty bad idea anyway, so really you'd only want to do this for a personal site. Except -- I can't think of any scenario where I want to edit a personal website, I have an Internet connection, and I'm on a computer I'm comfortable typing Git credentials into, but I don't have Git installed. Usually it's just going to be easier to make the edit locally and push to Git the normal way.
----
If you are trying to build a multi-user submission form:
You can build a purely static site on Netlify and use 100 hours of 'serverless' functions for free. Your functions can still do stuff like push changes to Gitlab and trigger Gitlab builds if you don't want to fiddle with a database, except now you can actually run real input validation. Plus, if you're using serverless functions, you can make your submission forms work without Javascript, which is a nice plus.
At the point where you're doing more than 100 hours of work a month running builds, you're also probably at the point where the pure-JS repo solution is going to have a really hard time scaling. And if you do reach the point where 100 hours doesn't cut it, you can scale up Netlify functions without rebuilding your entire submission form to use a different architecture.
----
None of that is to say that the setup Sammi describes isn't worth playing around with, or that you can't build a working site with it, or that people should be discouraged from trying. It's very cool; and it's useful to try things like this even just from the perspective that they're fun projects that will make you a better coder.
I just question whether it's really a holy grail, or even a good idea to hook up to your own Gitlab account. I would posit that the reason you see fewer people actually following through and adopting Isomorphic Git in this way isn't because no one's thought of it, it's because you can build the same solution more efficiently and safely for the same price ($0) using serverless functions or (worse-case) setting up a single $5 Linode VPN that you share between all of your sites.
You don't need to embed the git credentials in the site. I just made a login form that takes the gitlab credentials. Gitlab gives you 5 users per project, so you can give several people access.
You're absolutely right that this is not a replacement for a full blown CMS, or for when server side input validation is necessary, or for when different people need different edit rights. I've stayed away from using the term CMS for this reason as it implies that complex content management abilities are required. These "dynamic static" pages only give you full edit rights or not, and you need to build everything manually as it is a static website after all.
I have found this approach useful for hosting a static website for an artist where I wanted the artist to be able to easily update some info on the site. I imagine there are a lot of cases like this where you are building a small static website for someone else who isn't a developer and you want them to be able to update the site themselves later, so they don't have to bother you about it.
I wonder how this compares to wasm-git[1], which is libgit2 compiled to wasm for use in both the browser and node. At face value it seems isomorphic-git has been around for longer, and wasm-git seems hot off the press, but the latter seems like maybe it's got more eyes on the prize given libgit2's developer base?
(I have no affiliation with either project, just looking for options to use git in-browser and in node.)
The term has been weirdly adopted by the dev community in a way that kinda makes sense if you squint a little but has always created a dissonance in my brain due to my maths background.
But since we’re here! For those who don’t know the word from outside of the dev sphere, in maths it refers to having 2 way mappings between 2 structures. You can do it between totally seemingly unrelated parts of maths (you just need to show how to map between them). As a result, anything you’ve proved in one space holds true in the other. It can be useful if you have an intuition and understanding of one of the spaces but not the other.
I’m trying to think of a nice example but I haven’t thought about this type of maths for a couple of decades now. Answers on a postcard please!
> The term has been weirdly adopted by the dev community in a way that kinda makes sense if you squint a little but has always created a dissonance in my brain due to my maths background.
I don't really have a 'math background' (engineering graduate, but I don't think that counts, a mathematician certainly wouldn't) but I know (enough, roughly, layman's definition) what it means in mathematics, and work in software engineering.
I have absolutely no idea why this project is called 'isomorphic', it doesn't make any sense to me however much I squint, and doesn't seem to be explained anywhere.
I'd guess it's just an arbitrary name and isn't supposed to have meaning? Though it's a library for `git`, why not just call it `git`? shrugs
All the good names were already taken on npm / GitHub: git, git.js, js-git, es-git. "Isomorphic JavaScript" had just been coined and was trendy at the time (2017ish, see https://en.m.wikipedia.org/wiki/Isomorphic_JavaScript) and "isomorphic-git" is unique enough to be Google-able.
This makes me think that there should be two names in package management: a UID, and the import/use-it name.
Many such systems do work like that, but the UID is chosen by the author, so invariably it's a close match to the import/use-it name, and often it's actually the preferred name because the other had to be mangled to be syntactically correct in a program. Consequently, the main display name is the UID; not the import/use-it name.
The UID should be generated by the package manager, and only the import/use-it name chosen by the author. The latter would be the one prominantly displayed, and be allowed to conflict with other packages (that have different UIDs).
Even if you wanted to use two that shared an import/use-it name, there could just be a way of renaming them in the requirements/lockfile, e.g.:
[dependencies]
d34db33f = { version = "^0.1", name = "isomorphic_git" }
l333333t = { version = "^3.2", name = "native_git" }
where usually both of those use the import/use-it name `git`, and the names `isomorphic_git` & `native_git` are whatever I fancied for this project with this dependencies file - could've been `foo` & `bar`.
(Perhaps a more human-friendly format would be `git = "^0.1"`, and we only have to specify something like `= { uid = "d34db33f", ...` when the package manager detects that there is a naming collision. Of course the lockfile, if used, would already have the UID, in case a colliding package was suddenly created.)
The “isomorphic” comes in because the same library / API is available in both the browser and in node. So “anything you can prove in [node] is also proved in [the browser]” (in the context of this library).
Not that you're wrong, but it's a deep and pervasive concept in mathematics, and the field of computing is closely connected to mathematics in general. It's sort of a namespace collision.
Ok, but the same arguments could be made about the terms "refactor" and "plane" (as in control-plane/data-plane) but we seem to manage to cope with the overload.
This isn't an argument about consistency. It's an argument about whether overloaded terms are acceptable. If you want to argue that terms shouldn't be consistent that's an entirely different (and entirely bizarre, IMO) discussion.
Another term in computing/programming for "same code in different environments" is "portable". (e.g. C++ code that compiles/runs on both Linux/Windows). I've not seen 'isomorphic' used in computing outside of JS.
The community has been slowly moving to the term "Universal JS" for that concept, though. Probably partly due to the mathematical ambiguity, but mostly because "universal" is a bit more approachable term.
I remember my mind being blown when in Discrete Math 101 we learned that natural numbers are isomorphic with whole numbers. Was it, you map 0 to 0, even n to n/2, odd n to -(n+1/2) and all of us were like whoa and the TA is like "wait till I tell you about diffrent cardinalities" :D
This is just a bijection - for an isomorphism you need more structure. Roughly, having an isomorphism (which is a function) says:
0. I have two sets of things, and I can perform operations on each set
1. These two things are the same size (this is your example)
2. If I do operation A on elements (a, b) in set A, then perform the isomorphism on the result, this gives me the same thing as if I performed the isomorphism on a and b independently then performed operation B on their result
3. The isomorphism has an inverse with the same properties
[There are various types of isomorphism depending on what part of maths you’re thinking about. But they all have roughly this idea of “same size” and “preserves something cool that I can do in each of the spaces”]
Yes, I probably ought to have mentioned that. Though ime people are more likely to talk about isomorphisms in other categories, since the only “structure” a set has is its size
Bijective means that a mapping is exactly one to one.
Homomorphic means that "structure is kept". E.g. in a vector space you can add and multiply by a scalar, so a mapping f is homomorphic if for all x, y in the vector space and lambda a scalar
A little correction: an isomorphism is a homomorphism which is bijective and has a homomorphism as its inverse. For many structures the inverse being a homomorphism automatically happens as a logical consequence of the function being a bijective homomorphism, but this isn't the case for all structures. The most prominent examples are topological spaces: a bijection between topological spaces can be continuous (i.e. a homomorphism of topological spaces) but can have an inverse which is not continuous.
The classic example is a function which maps an interval to a circle by applying cos and sin to get the coordinates; this is continuous as small changes in the interval result in a small change in the position on the circumference, but its inverse is not continuous because there will be a point on the circle where the nearby points on one side come from the start of the interval but the nearby points on the other side come from the end of the interval.
From the project: "The "isomorphic" in isomorphic-git means that the same code runs in either the server or the browser. That's tricky to do since git uses the file system and makes HTTP requests. Browsers don't have an fs module. And node and browsers have different APIs for making HTTP requests!"
I'm wondering if I can use this in an ERP system I'm developing. A feature request has been to see a diff history of actions on an entity (say, Purchase Orders). Using something like this, I believe I could make a 'commit' on each save, then display a history of commits to the user to 'diff' against to see previous revisions (who added which items, when did the PO number change, etc).
I was planning on storing this history in the database as raw JSON, then diffing the JSON server-side, but this may provide a more robust solution.
That could work, but note that there are other libraries that are purpose-built to do diffing without the rest of git's many features. We are using jsondiffpatch [1] and quite happy with it. It also has builtin formatters to output e.g. html view of the changes.
I am personally using Isomorphic Git exclusively in my automation scripts (written in Node) where it is about infinity times more convenient than shelling out to Git and parsing its output (something which feels dirty to me even in scenarios where it works, like in VS Code's version control pane which does this), however I know a few folks are looking at online IDEs and various things with P2P - like using WebRTC to mesh up clients and using Git in a truly distributed fashion across this WebRTC mesh. The fact that you can supply your own fs implementation makes these kind of applications really easy.
Let say you are building a document editor web app, or any app for creating content. Instead of simple undo/redo you could leverage git powers for versioning + tagging + branching + collaboration on document content. I needed this kind of functionality on one project, but I haven't yet looked into isomorphic-git to see if it is actually a good fit for such purpose.
Pretty much this. I’m building a WYSIWYG note taking app that syncs with GitHub, sort of like Prose.io but more focused on the note taking than CMS part (although the option of publishing notes would combine these two).
I think there’s two benefits of isomorphic git for this use case.
1. It’s more api agnostic: if I wanted to support Gitlab, I wouldn’t have to write substantially more code tailored specifically for Gitlab’s API because it speaks git. You would only have to switch out Github's auth for Gitlab's auth. I also found that add multiple files and versioning them with Github’s API is a pain; they’re remaking a client called Octokit but it’s pretty underdeveloped right now.
2. Offline version control: you shouldn’t have to be online to take notes. With isomorphic git I can add and commit, and then push back to GitHub later when I have Internet. Fun fact: to get git to work on the browser, the author of isomorphic-git wrote a library to port NodeFS commands backed by IndexedDb. This means you can also have offline image drag-and-drop.
There are some minor issues with isomorphic-git when I checked a month ago.
- Largest one is only fast-forward merges are supported; not sure how to get around this one besides cloning the remote Github branch and maybe manually merging files.
- Another one is a bug with git init followed by git add remote and git pull which leads to a reference error because no local commit has been made yet; you can imagine a user who immediately logs in with Github so you want to sync with a pre-existing repo.
Ok - but isn't the hard part of collaborative authoring the conflict resolution not the storing of versions per se. Git has a merge mechanism, but it's not necessarily that user friendly?
Isn't something like OT ( operational transforms ), or CDRT ( Commutative Replicated Data Type ) a better way to do this?
Conflict resolution of course depends on actual content. Source code could be among most tricky ones to auto-resolve or detect when manual merge is necessary. Documentation, art and other not-so-strict content could be more auto-resolvable than source code. Git is here simply a mechanism for allowing different work flows. Your web app should enforce best practices for non-expert users. The added benefit is developer familiarity with git concepts.
I wasn't familiar with OT and CDRT, so maybe I'm off the mark. They seem to try to solve real-time collaboration between multiple users editing same part of same document. This would always require some meta-document or meta-channel where users coordinate what should be changed, in what way and who does what. I would argue that git model is better: user decides when to pull and when to push changes, and the communication between collaborators is outside the scope of the versioning system.
Collaboration is just one application. Even single-user content creation could vastly benefit from built-in versioning. This would eliminate some mental burden and usage friction (tagging), and open up more possibilities to experiment with different directions (branching). The feeling you get when you check-in working version of your code and you are now free to break stuff, that kind of feeling is missing when creating slides, graphics, music, CAD design...
Further - I am using git commits to store hashes for code-created artworks, so that I can reproduce and remix the exported PNG at a later point. See here:
Fork-able web content (e.g. blog posts, class notes) is something I've been looking at, but you would still need to rely on an online service like Github or Gitlab.
I’d love to see some work put in to making this work with React Native. I spent a day trying to get this to work a few weeks ago with no joy, and I couldn’t find any indication that anyone else had managed to make it work either.
My particular use-case is that I'm not a Swift/Objective-C developer, I'm a JavaScript developer.
React Native opened the door to allow me to build apps and, realistically, I'm not going to spend the time battling through trying to get native git working through React Native. If there's a native developer out there that wants to tackle this problem and wrap it up in a nice JS module then I'd be in your debt!
Depends what users, if you're targeting some education format or institution that wants some locking down of their OS git native is kind of a non starter. No one can configure it for you, there's 65 million install options and you're loading a bash shell into windows.
Very different from "this app has packaged tutorials and tools just start at lesson 1"
This would seem like a workaround but it's possible to run node.js on Android and iOS and bundle it with your react native app so that might be a potential way forward: https://code.janeasystems.com/nodejs-mobile/
I'm not talking about making use of some "serverless" backend like Firebase or something. I'm talking about only using git hosting and static website hosting and in browser computation to make "dynamic" websites.
What you do is:
* You make a regular static page with whatever static web page framework or view template software you like that runs in the browser. I got Handlebars to work. (I'm looking at trying Choo next, because it is literally one hundredth of the size)
* Make another "admin" page where the user can input the data. This page first checks out the current git repo.
* When the user hits submit you use the newly edited data to build the static site in the browser and commit/push everything to git where it is automatically pushed to netlify.
You first have to bootstrap this process on your own machine in the first commit.
I've been holding on to this secret for over a year. Well the cats out of the bag now. I haven't run into someone doing something similar with Isomorphic-git yet. I've had it in production for an artist website for almost a year. I don't know if this is good or bad for Gitlabs or Netlifys business model.