The performance improvements are very noticeable : general page load times, pushing code to gitlab is as fast as github now and the commit messages load insanely fast, compared to a few months ago where loading commit messages in the UI used to take at least a couple of seconds. I don't follow gitlab development too closely, but they seem to have focused a lot on improving cache performance.
But one change that I did not like was the the removal of the side panel which could be pinned. Navigation required fewer steps, but now an extra click is required to bring up the options.
Git access will be improved now that we have Gitaly as part of this release. In GitLab 9.1 we want to move git access from the application server to the file server. That should greatly reduce the latency.
I'm glad this is a priority for you guys Sid, the one thing keeping me from using Gitlab is that it doesn't feel as snappy as Github. But as a point of reference, I'm also a guy that won't use Atom because it doesn't feel as snappy as Sublime. For some of us, speed comes first.
Note that Atom is written in Javascript, so the run time is extraordinarily CPU heavy. Even though many Electron (Javascript desktop framework) apps may have snappy UIs, they're typically even worse than Java for CPU (battery life, etc).
It's all relative... what kind of computer are you using, the cpu speed, memory available, even OS and drive/ssd can make differences. Even on an i7 with 32gb and an SSD atom was too slow for a really long time... in fact after trying earlier releases of atom and brackets, I almost didn't even try VS Code. I'm really glad I did though.
In any case, it's just a matter of relative performance and perception in relation to what one runs on, and their needs.
No, I meant that VS Code, (which is similar to Atom in both features, technology used and price), is (much) faster than Atom, (Electron vs Electron), it is obviously not faster than Sublime, (Electron vs Native).
I understood the OP's question as what is similar to Atom, but faster, is it VS Code?
Taking a look at Gitaly & the conversations around it, I don't see much in the way of consensus. Are you using Redis for the distributed lock or is this still a centralized system?
My query being you can effectively use the native pre-receive hooks & wrap the git shell to allow consensus without needing to replace git with an RPC system like Gitaly.
Gitaly is still a centralized system. It shards the git repositories over multiple git servers in one datacenter.
We looked into Git Ketch https://www.infoq.com/news/2016/02/google-kick-starts-git-ke... for a distributed system based on hooks. But this requires a sync per repository (very slow with 100k+ repos per server) and it is no longer actively being worked on by Google.
There are plenty of off-the-shelf consensus algorithms/services you could use in the git-hooks that don't have the git-ketch limitation. Zookeeper, etcd, consule, redis, etc. It really just matters your criteria and requirements, they all of trade-offs but I'm sure one fits.
To be frank I think your time is better spent looking into why one of these consensus packages isn't the right call for your needs. Since they could easily plug into the git-hooks & git-shell commands, it could avoid a massive investment of time from your team and you.
Next time you think about distributing git, you should take a look at the combination of jgit and a distributed key value store.
A little while ago I spent a night and I was able to get a fully distributed git http server up and running. I used HBase and was able to get it most things working. That allows the only contention to be around refs. Everything else is well distributed and spread across as many machines as you need. Caching is handled on the key value store side, and the http server side. I didn't get to git gc work though.
> That allows the only contention to be around refs.
This is solved without modifying the git client using a distributed/shared lock wrapping the git-shell & git-hooks (pre-receive) . If you were to modify the git server behavior to say use epaxos, one could provide even better latency guaranties https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf .
I'm not sure about the GC operation though, did you use the DfsGarbageCollector that is part of Jgit? I thought that was provided as part of the DFS interface.
> you can effectively use the native pre-receive hooks & wrap the git shell
The way GitLab is organized (pre-Gitaly) makes it very hard to do things like that. Roughly speaking, when handling a push, it is a coordinated dance between GitLab components to make sure it all goes in, but none of these components has full control over the push from start to finish. One of the reasons we are creating Gitaly is to create a 'place' (namely Gitaly) where we _can_ exercise such control.
The other big reason for building it is not having to use Git repos on NFS.
Thanks. Most of our monitoring is public on http://monitor.gitlab.net/ the things I showed are not because we're still working to make Prometheus work with multi process applications. I'm not sure the Json is public.
Sorry about that, we've replaced the links with screenshots.
The problem is that we can't measure Unicorn timings with Prometheus since it is multi-process. Our Prometheus team is working to fix this for everyone.
Thanks for the feedback regarding the sidebar navigation! Here's the issue that we used to chat extensively about it and came to our current design here in 9.0: https://gitlab.com/gitlab-org/gitlab-ce/issues/26200
There's lots of discussion there. We went with a design that leans on getting out of your way and recovers some screen real-estate. We do recognize the one additional click required though. And we are actively listening to feedback and will continue to iterate.
The application I'm working on, I went with a 3-state navigation... null (default), true (open) or false (closed) ... at large screen sizes null is a soft open, smaller soft closed... toggling goes between the respective hard state and the null/default state. At sm-medium, it's an overlay, at phone/xs it takes over the screen.
I find that this is more intuative to use than a hard on/off only.
I agree with the parent about the sidebar. I already disliked the change to make it hidden by default with the pin option since it was a bit inconsistent for me, but it at least gave the option for best of both worlds.
Ideally this should be an option. I get that 'modern UX' is all about more screen space and whatnot, but I'm not a fan of that trend personally. It seems that all that's been gained from that change is more empty space at the side of pages. Personally, I'm the type that would rather have more information and convenience on the page than extra space.
Would it be that hard to add an option to enable the side bar if people want it? That's what I like about GitLab, that it's very configurable.
Trading usability for screen real-estate? Sounds like your designers have gone off the deep end and fail to realize that Gitlab is a productivity tool, not an art piece.
That's quite a bold statement from an armchair UX designer. They obviously have reasoning and supporting data, maybe you're in the minority of users who've found that useful?
Thanks for the feedback! In this iteration, we think recovering more space helps users focus on content, and allows them to see more content at once. Definitely we recognize the tradeoff of additional clicks and discoverability as a result.
We are definitely focused on usability, and navigation in general. We don't claim to know all the right answers all the time. So we are iterating with each release, and working hard doing user research as well to inform our decisions. You can see some of our recent work regarding nav here: https://gitlab.com/gitlab-org/ux-research/issues/3. Thanks.
I'd argue that extra screen real estate for the content that matters often lends itself toward better UX, but any specific suggestions for improvements would be appreciated. :)
Can we just all agree to keep the damn sidebar available - if UX wants to remove it, fine, as long as users have the ability to easily permanently pin it back open if they wish to do so.
By now GitLab UX should realize that enough developers find great utility in that sidebar.
I sincerely apologize if the tone in my post offends anyone - I let my frustration with this issue get the better of me while writing my comment.
However, I do stand behind the main points of what I wrote:
1. It's disappointing and frustrating to see what is almost the same issue coming up only 9 months later (please look at the link I shared above), where now I'm seeing all the same arguments in favor and against the side bar in these comments as well as in the new related issue thread. We are going through almost the same exercise again.
2. On removing the sidebar - allowing the sidebar to appear and be pinned would be a more inclusive UI/UX choice. Please allow us to pin it again.
I will also add that over the last 1.5 years we've been using GitLab EE the GitLab team has been awesome. I'm continually amazed at how quickly many feature requests become part of GitLab.
Yeah I miss the sidebar too, it annoyed me and our devs when the pinning was broken and then when there were less items in it.
It felt like without the side bar there is wasted screen realestate that could have project related options or navigation in the side bar.
As resolutions and screens get larger I'm a strong believer that as long as you don't over complicate or make the interface too busy, you shouldn't waste ⅔ of a web site with white space.
Regardless, very happy with 9.0 otherwise, another fantastic release from the team and contributors.
I love and use Gitlab and am excited about the 9.0 performance improvements.
On the other hand, I found it absurdly difficult to get feedback or action from Gitlab on an MR. There was an MR that fixed a critical problem in the virtualbox runner, and it sat there for about 5 months with no response to either the original submitter or my emails to gitlab/gitlab ci maintainers. About a month ago I finally decided to abuse your customer support addy to ask if somebody could force the maintainer to respond before the code rots. By that time the original submitter (who in comparison turned out to be easy to contact and collaborate with) was long gone. And another user had already announced a separate script they developed and maintained to work around the lack of merging this fix.
I'm glad it finally got merged, but it was only after deciding to use my free time and treat the process as a kind of game to be beaten. But that's a one-off game I won't play again. Plus I doubt the original submitter, a Go developer, will be making another merge request.
As Kamil already wrote, the number of notifications, issues and merge requests that we follow is enormous and sometimes something may be missed. I'm doing my best to have eye on everything but this is not always possible.
From my side I can only say that I'm very sorry that your work and work of all people engaged in this merge request was noticed so late. We're working on improving responsiveness in GitLab Runner project so hopefully we will avoid such delays in the future.
Tomasz,
developer at GitLab,
GitLab Runner project maintainer
Thanks for writing that comment and making us aware of that case. I'm very sorry that it did that long to get reviewed and merged. The number of issues and merge requests that pass our hands is just crazy and sometimes it happens that some of them are oversight and not properly scheduled. Having people to be persistent, like you, makes them finally to be merged and be part of the product. Thank you for doing that! From our side, we will try our best to make this process to be more efficient.
One point may not have been clear-- this isn't just a matter of a single bugfix being available to the userbase sooner rather than later. Rather, it is that multiple iterations through a development cycle were missed. From my rough recollection:
1. There was a problem setting up an OSX guest to work with virtualbox runner on OSX with passwords. We worked around this using pubkey-based auth between guest and host.
2. There was also a timeout issue mentioned on the tracker for which Sam found the relevant code and suggested (but did not create) a fix.
3. Sam seemed to think a snapshot-reuse approach was too complex and likely to hang or fail. I didn't agree and was trying to explore ways in which they could run reliably as it cut CI time in half for my use case.
4. For uploading artifacts, the OSX guest's gitlab-runner binary had the wrong name requiring the user to manually create a symlink with the correct name. (I think this is fixed in 9.0, but not sure since I still have the symlink there and it's still working.)
At the time my CI builds (and probably Sam's) weren't working at all, so we both ate the cost of learning how the CI build system worked in order to debug it. If there had been feedback saying that his MR was the right (or wrong) approach I would have been happy to apply my knowledge gained at the time to the related problems mentioned above in order to debug and eventually fix them.
But now that knowledge is gone. The cost for me to regain it outweighs the benefits of small improvements to what is atm a working if imperfect system.
We've been using gitlab for most VideoLAN projects, and are moving towards it for all of them (including vlc and x264). A lot of things are great, and going in the right direction, but the issues tracker is really lacking.
The lack of custom fields, that you find in all other bugtrackers except github, and the lack of dependent issues are really blocking us to move full to gitlab.
And the biggest issue is that they're discussing to do it, but only in the non-open-source version, which we cannot use...
I can't promise anything regarding these features. Especially custom fields are a typical EE feature in our perspective. The dependent issues feature proposal https://gitlab.com/gitlab-org/gitlab-ce/issues/4058#note_249... is still on the CE tracker but I'm not sure what it's future is.
Both of these features are not scheduled for development yet. We scope work out month to month, so we can't promise anything regarding those at this time. But we are discussing them as you see in the issues and considering how they would fit into our product as we recognize they are important features for many teams.
It has a best-in-class code review tool and a bug tracker with custom fields, permissions, templates, dependencies and more.
Its code review is built on a patch workflow (like the email-based one that the Linux kernel uses) and works very well, especially for large teams. Merge requests are clunky in comparison.
Many large open source projects moved to it, including Blender, Wikimedia, FreeBSD, Haskell, LLVM and more.
I'm using it in production, am glad to answer any questions.
Our EE version is publicly viewable but not open source. I can't speak for the VLC project but previously they indicated that cost was not the problem. They care about the free as in freedom instead of the free as in beer.
Congratulations on shipping 9.0. It's great to see a lot of improvements. Being able to reorder issues in the board is a welcome change. The subgroups is an interesting feature. I look forward to playing with it more.
I am still waiting on the ability to view issues on a board across a group (or now, a subgroup). Since Gitlab projects are repo-centric, I have hard time tracking issues across many repositories in a single place. A board that is visible at a group-level would solve this problem.
I believe this is a feature that's coming soon, though. I am looking forward to it.
This seems to be the huge blind spot on gitlab product design. Everything is tied to git repo's while in the real world installs, users want global views of things which go across git-repos. I suspect this comes from starting off as a github clone, which has it's own constraints as a product (and it's public face).
Hi, we are moving more and more into the direction of going multi repo/multi project. Subgroups is the most significant step forwards for this. Expect to see cross project pipelines, management feature, monitoring etc
Like Job mentioned, we've been ramping up issue boards and groups (subgroups now) rapidly in the past few releases. Group-level issue boards are definitely on our radar. Feel free to leave any comments on the issue itself that Job linked: https://gitlab.com/gitlab-org/gitlab-ee/issues/928.
Currently we are considering bringing all the issues of all projects (that are in the same group) into one location, i.e the group-level board, exactly as you describe it.
Team-first/team-based collaboration is something we are now highly interested in, with subgroups being a big feature in 9.0 in this direction. Our next step will be to tie that with issue boards. Thanks for the feedback!
When you say team-based, do you mean as in organizations? That would be a huge killer feature. I've only been part of an org on GH, but org wide issues/boards/communication is a huge missing piece.
Not that this is a solution everywhere, but I'm always telling people that single-repo is the way to go for an integrated team. My rule is that if your business unit can reasonably control a codebase, and you have access to it, it should be in the same repo as your other codebases, full-stop.
I've seen too many npm messes where people obsess over putting every button, dropdown, heading, and dumb image component in its own company-specific repo. Then they put the whole thing in a "UI kit" which realistically they will never reuse.
We originally built GitLab with that use case of a single repo in mind. Create a GitLab project, and you have a single repository for code. We know this is an important use case for many teams, because of it's straightforward, simple scenario. It's something that's important to us, even as we expand to other use cases of multi-repo teams (groups/subgroups housing of projects). Thanks for the comment.
Yup, I think Google & Facebook have this one right, a mono-repo is the way to go.
But there are advantages to multi-repos, especially if you're trying to share with clients and/or open source. You can have your cake and eat it too: we use https://github.com/splitsh/lite to update our micro-repos on every commit to our monorepo.
We're working on it! We were holding off until subgroups shipped so we can make sure group-level boards work well with it. It's on the shortlist for one of the next releases. Our PM for this, Victor, might provide more detail here later.
Gitlab was really promising for my colleagues and I, but its inability to host git repos on the url root (aka gitrepo.mycompany.com/product.git) without hacking means its a no-go as we're not particularly inclined to redo all our development, deployment, and testing infrastructure. We use git a lot in our organization, including customer-facing stuff, so it would be a large transition.
Also stubborn coworkers who don't want to monkey with git settings.
Which is a shame, I really like gitlab and would love to replace gitolite :/
Just curious since I am not experienced with this stuff, are you talking about a TCP reverse proxy (so it could work with the SSH protocol too)? And would haproxy work equally well to nginx for this?
If I remember right, nginx doesn't support ssh + http(s) on the same port. I'd be happy to learn this isn't correct, though because my personal server only is allowing http through.
As much as I love GitLab the UI/UX is still very weak compared to GitHub. And the sidebar change is something in the completely wrong direction. You should also really check out how it looks on a 4k screen.
They've set a very high bar for GitLab because not only is GitHub a great repository and project UI, it is one of the best overall application interfaces out there.
It's difficult being compared with GitHub and bucketed with them from a UI perspective, but GitLab really are holding their own and seem up to it
Well, IMO the Github UI has been taking some steps backwards recently. If Gitlab keeps improving and Github worsening, they'll be at parity before too long
I imagined you'd meet in the middle, rather than at the bottom!
I like what you guys are doing - my comment was more meant as a well-natured jibe against the recent Github changes, which I'm really struggling to like (the black header a few weeks ago, and today the search results interface).
Thank you for the feedback. The sidebar is indeed oft-discussed. And we don't claim we have all the right answers. We work hard every iteration to make the best decision, ship updates, and get feedback.
We are working hard to improve UX and navigation specifically at GitLab. Our UX team is focused on design and user research. Here's some work on the research end: https://gitlab.com/gitlab-org/ux-research/issues/3.
We also have a new research panel, which includes usability texting. Anyone is welcome to join to provide further feedback, which will significantly impact our product development.
What convinced me to try out gitlab today: "Today with 9.0, we are excited to release Deploy Boards for environments running on Kubernetes. The Environments page of Pipelines now offers a single place to view the current health and deployment status of each environment, displaying the specific status of each pod in the deployment. Developers and other teammates can view the progress and status of a rollout, pod by pod, in the workflow they already use without any need to access Kubernetes."
Great to hear that. Deploy boards is one of the few features that came from our own vision more than customer requests. So we're very happy to get validation that this is something people need. GitLab 9.1 will have canary deploys and an automatically deploy board to make deployments even better.
Is there an api for users that might want to integrate alternative schedulers for these features? Would be lovely to be able to integrate Nomad somewhat natively.
No, but we're hoping to work together with the community to make all these features available on all major platforms [0].
One thing that we're seeing is that most schedulers are moving towards supporting Kubernetes, which goes a long way in making this work there. Right now we support Kubernetes through Google Container Engine or via Openshift.
Right now Kubernetes is a project service in GitLab. We would love to have a project service for Nomad to do the same. Right now we're focussed on Kubernetes but we hope the wider community will pick up the other schedulers.
Also see https://about.gitlab.com/strategy/
"The people that have the most knowledge have to prioritize breadth over depth since only they can add new functionality. Making the functionality more comprehensive requires less coordination than making the initial minimal feature. Shipping functionality that is incomplete to expand the scope sometimes goes against our instincts. However leading the way is needed to allow others to see our path and contribute. So when in doubt, the rule of thumb is breadth over depth, so everyone can contribute."
Why are Deploy Boards being restricted to EEP customers? I would love access to that, but I don't need the support and geo-replication (and 5x cost increase) that comes with EEP.
Really glad to hear that! We're really excited about this direction and think it'll be a game changer for DevOps. Of course there's more to come. Take a look at our direction[1] and join in the discussion!
Any documentation / talks / videos about the gitlab + kubernetes integration ? Thanks for the awesome product. At work we switched to gitlab for new projects and loving.
The video in the blog post is all on Kubernetes as well, ans shows off terminal, deploy boards, and monitoring; all on Kubernetes. It doesn't go into detail, of course, but it's great if you just want to see it.
I like Gitlab, as an absolute noob I'm not really ready to make every mess of a program I use public (as Github requires). I fear that it can and will be used against me. I'm also such a small time user I think paying for Github is too much for my 3 > 200 lines projects. And so you end up at Gitlab. It works very well for me. If I want to, in the future, I can host it myself which is a nice thing. From this update I most like the dark-themed code view, yes I'm a simple man that is just learning how to code.
I would love to upgrade on my GitLab server, but I haven't been able to build GitLab with the omnibus builder since 8.14. I opened up a ticket but it hasn't garnered a response.
It seems the issue is on the wrong issue tracker. Our omnibus lead Marin will make sure that it lands in the right place and gets some attention (see the comment from ZJ).
With the focus on performance, is there any chance we could get pagination on the deploy keys page? Our company is self hosting the CE version and we currently have 339 deployment keys. Every time we need to load the deployment keys page, it takes upwards of 20s. We could probably organize things differently and cut that down, but that would be a significant restructure and time investment for what would essentially be a band-aid. I (and others) have requested it before, but was told it was too low of a priority and it wouldn't be worked on.
So they are on the main Project Settings page as of 9.0? If that's the case, we will avoid upgrading until that is changed because that page will become fairly unusable in our case, both in terms of speed and general UX because our Deploy Keys page is currently 20,777px tall. That is a lot of scrolling to have to deal with.
I will subscribe to those issues and hopefully it will improve. As of now, though, that is a complete non-starter for us.
Pagination on the list of commits in a PR would be awesome also.
Oftentimes a week of work that's bundled in our develop->master RC merge will have more than 100 commits. Gitlab _only_ displays 100, and doesn't let you see the rest; since I like to skim down the commit messages to make release notes for the RC, this is pretty inconvenient.
The previous suggestions were done on the old suggestion site and I cannot find a link to that site anymore; it's been a while since I have personally brought it up but noticed all the mentions of performance improvements and, combined with our ever-growing list of keys, it felt like something to bring up again.
If you host it on top of a Kubernetes cluster, yes. Github (including kanban boards) + CI + Heroku (including deployments, workflows) + now even Monitoring.
They are selling what one might call an "Omakase" experience for the whole SDLC. They call it "Idea to Production".
That is a very apt description, thank you for that. We're really inspired by the Rails doctrine http://rubyonrails.org/doctrine/ and think that the Software Development LifeCycle (SDLC) has room for a solution with convention over configuration.
We have open source self-hosted (Community Edition), open core self-hosted (Enterprise Edition), or hosted (GitLab.com). It's more like GitHub + Chat + CI + CD + Heroku + New Relic (early stages). Where Heroku forces you to use their platform, GitLab allows you to use any platform that runs Kubernetes.
```
Running with gitlab-ci-multi-runner 9.0.0[...]
Cloning repository...
Cloning into '/builds/xxxxxxx'...
warning: You appear to have cloned an empty repository.
Checking out xxxxxxx as master...
fatal: reference is not a tree: xxxxxx
```
The new enhancements seem very promising. We've been using gitlab for our internal use for a while now.
Having said that, every time a new gitlab release comes around, I always looks for updates in one area expectantly but mostly left unsatisfied. And that is, separation of the concept of Project from "one git repo"
E.g., I want to manage a project that ties in issues from various different git repositories. It's pretty common. Almost no actual "project" in a real life business is limited to just one git repo.
I want a system-wide wiki that collects knowledge about the entire operations. The idea that each wiki is tied to a particular git-repo is.. just... wrong.
Heck, even the new search feature across projects somehow doesn't include wiki's. How did that decision get made?
I run my fleet of runners in pre-emptible instances on GCP (they die after 24h).
I forgot to pin the version in my startup script, so all it did was a simple apt-get install gitlab-ci-multi-runner.
For some reason, version 9 of the gitlab runner cannot connect to our Gitlab (it fires a 404 on register).
Anyways, look like 9 is a cool release, but we won't be upgrading until the sidebar comes back (did a little poll in the team and it 100% was in favour of the sidebar)
The reason is that in 9.0 we've prepared a new API for Runner requests (as part of new v4 version of API) and Runner 9.0 is using only this version.
Version v1 of the CI API is still supported by Runner 1.10.X and 1.11.X. On GitLab's side it will be supported until August 2017 and until then we will also support Runner 1.11.X.
What do people use to keep up to date with all the non-stop Gitlab releases? I used to use sandstorm but their port is very outdated by now (same goes for bitnami). I switched to cloudron and it has served me well but I am always on the look out for other solutions. On a side note, if you use the omnibus packages, are you a full time (or even part-time) sysadmin?
Have you just setup a cronjob for backups? And do you also test somehow that those backups work (or rely on gitlab doing the right thing?) Or maybe you use server snapshots? Also how how did you go about SSL?
Glad to hear that. You can also import your projects directly (everything will be copied over).
We don't see any reason for you not to do so. GitLab should be the easiest place to get started. If that means logging in with GitHub, we're happy to make that possible.
I wonder if this will provide a benefit in a resource constrained environment. I run the official Gitlab Docker/gitlab-ce container in a host with a J1900 processor and 4GB RAM and it is barely usable. Most pages timeout on the first load.
At present it looks like it hasn't been updated since 2015.
Currently running gitlab over docker on a dedicated 8 core system with 32G of ram, it routinely uses 9G of that for barely 30 users. (it's the only thing running on that physical host)
Sorry just to be clear - I really like GitLab - and I feel GitLab tries hard to make a great product for their users - and clearly they have a good understanding of the development community. It's just that I felt this issue had already been discussed and resolved - to be yet again undone. So just feeling frustrated we have to go through this again.
Any idea how I can upgrade my docker installation on Kubernetes? At the moment in my Helm chart i have specified exat Gitlab and Postgresql version. Do i need to bump postgresql too when changing Gitlab?
Most of CD including deployments already works for private projects, it is only the auto deploy https://docs.gitlab.com/ee/ci/autodeploy/index.html that doesn't work yet. We would all like to see auto deploy working with private projects and it is scheduled for 9.1
If you don't want to use GitLab.com to host your code you can install GitLab yourself or use the mirroring function in GitLab to deploy code hosted elsewhere.
will just wait for "Gitlab.com" to get the functionality - is there any lag between Gitlab version upgrade and "Gitlab.com" using the upgraded Gitlab version ?
Will wait it out for Gitlab 9.1 :) Thank you for the amazing work.
But one change that I did not like was the the removal of the side panel which could be pinned. Navigation required fewer steps, but now an extra click is required to bring up the options.