what do y'all use for CI/CD?
from chrash0@lemmy.world to selfhosted@lemmy.world on 16 Dec 10:33
https://lemmy.world/post/40288876

hey nerds! i got a lovely email from GitHub this morning that their increasingly vibe-coded, barely-working Actions features are about to get more expensive (charging by the minute for something that notoriously spin-locks is a special flavor of shit sandwich).

i usually just use whatever i’m given at wherever i’m working. i do have a project that i maintain to parse Ollama Modelfiles tho: github.com/covercash2/modelfile and to be honest, Actions is the only solution i’ve ever used that came close to sparking joy, simply because it was easy to use and had tons of community mind-share (i’ve definitely heard horror stories and would never stake my business on it), but this price increase and all the other news around GitHub lately has got me side-eying self-hosting solutions for my git projects. Forgejo seems like the way to go for git hosting, but Actions in particular Just Works™️ for me, so i’m kind of dreading setting something up that will be yet another time sink/rabbit hole (just in time for the holidays! 🙃).

i can install most of my tooling with my language toolchain (read: rustup and cargo) which makes things fairly neat, but i just don’t have a sense for what people use outside of Jenkins and Actions.

i thought this community might have some insight beyond the LLM generated listicles that have blighted modern search results.

thanks in advance 🙏

#selfhosted

threaded - newest

irmadlad@lemmy.world on 16 Dec 10:47 next collapse

Watching this thread because CI/CD is something that I’d like to get into.

Zagorath@aussie.zone on 16 Dec 11:50 next collapse

Ditto

elephantium@lemmy.world on 16 Dec 21:25 collapse

Are you a programmer?

irmadlad@lemmy.world on 17 Dec 06:23 collapse

I…uh…I pretend I am from time to time.

eager_eagle@lemmy.world on 16 Dec 10:50 next collapse

fwiw, you can self host a GitHub actions runner

Natanox@discuss.tchncs.de on 16 Dec 10:54 next collapse

Don’t they want to monetize those as well?

chrash0@lemmy.world on 16 Dec 11:00 next collapse

yes, according to this morning’s email

eager_eagle@lemmy.world on 16 Dec 11:14 collapse

ah right, my bad

scrubbles@poptalk.scrubbles.tech on 16 Dec 11:58 collapse

But you are charged for it.

Carol2852@discuss.tchncs.de on 16 Dec 10:57 next collapse

I’m using gitea which has CI compatible to GitHub actions with my own runner. It’s pretty straightforward to set up and didn’t give me any headaches yet. It’s a very small instance just for my ownaybe dozen projects though.

yaroto98@lemmy.world on 16 Dec 11:01 next collapse

This is what I was using till I switched to forgejo and never got around to setting up one of their runners.

cecilkorik@lemmy.ca on 16 Dec 12:44 next collapse

If it helps motivate you to give it a shot, I found gitea’s runner very confusing to set up, but I felt like forgejo was better designed, pretty easy and well documented.

chrash0@lemmy.world on 16 Dec 12:56 collapse

heck yeah this is the review i was looking for 💯

Carol2852@discuss.tchncs.de on 16 Dec 21:27 next collapse

I run their act binary on one of my servers. Can’t remember much of the setup, so I can’t be too bad. I did have to change the used images though, but I guess that comes with maintenance of you own runner anyway.

finn@toot.fan on 16 Dec 21:34 next collapse

@yaroto98 @Carol2852 Same here, switched from gitea to forgejo. I still was using the act runner for some time, later I replaced it with the forgejo-runner.

Works pretty smooth!

witten@lemmy.world on 16 Dec 21:48 collapse

When you switched, did you lose all of your Gitea data? Or was that somehow importable?

finn@toot.fan on 16 Dec 22:01 collapse

@witten I have switched maybe a year ago or something like that. Didn't loose anything because I was running a compatible version at that time:
https://forgejo.org/docs/latest/admin/upgrade/from-gitea/

If you are running a recent version it's probably a bit more complicated.

witten@lemmy.world on 17 Dec 10:04 collapse

Ahh gotcha, makes sense.

witten@lemmy.world on 16 Dec 21:47 collapse

Out of curiosity, how did you switch to Forgejo? I thought Gitea and Forgejo have diverged to the point where you can no longer just switch over without losing stuff.

yaroto98@lemmy.world on 17 Dec 06:10 collapse

I hadn’t used gitea for long. I just had both running, and then cloned my repos one at a time manually. So long as I had the code, I didn’t really care.

witten@lemmy.world on 17 Dec 10:04 collapse

Got it, thanks.

chrash0@lemmy.world on 16 Dec 11:02 next collapse

good lead. it’s just the one project for now, and to my surprise it’s actually a dependency for the ollama-rs project, so i feel somewhat obligated to keep it stable.

corsicanguppy@lemmy.ca on 16 Dec 13:21 collapse

CI compatible to GitHub actions

Ugh. More yaml?

chocrates@piefed.world on 16 Dec 20:25 collapse

I get the hate but did you ever have to maintain jenkins pipelines? I’ll take yaml any day.

I’m game to explore the next evolution though.

Carol2852@discuss.tchncs.de on 16 Dec 21:24 collapse

That was my first thought as well. 😁

michael@piefed.chrisco.me on 16 Dec 11:02 next collapse

Used to use travis or clicleci and they both worked really well. Theres some issues with travis being old/expensive and circle got in touble for a few security issues though. gitlab has some nice tools from my experience.

Im interested as well. Ive got a forgjo that I would love to hook into at some point.

[deleted] on 16 Dec 12:20 collapse

.

eager_eagle@lemmy.world on 16 Dec 11:14 next collapse

btw, the prices of managed runners are going down, not increasing

docs.github.com/en/…/actions-runner-pricing#stand…

still good to have a self-hosted alternative though

chrash0@lemmy.world on 16 Dec 12:11 collapse

i honestly didn’t look that close, obviously haha

but yeah, i’ve been kinda looking for a reason to de-Microsoft my stuff

Mihies@programming.dev on 16 Dec 11:18 next collapse

I use cake build to create the build script and then I can run it from wherever.

Jayjader@jlai.lu on 16 Dec 11:27 next collapse

Forgejo has their own runner: forgejo.org/docs/latest/…/runner-installation/

I’ve used it on my personal machine, was very easy to setup and mostly compatible with GitHub actions out-of-the-box (including things like actions/checkout@v4).

scrubbles@poptalk.scrubbles.tech on 16 Dec 11:58 next collapse

Forgejo runners are great! I found some simple actions to do docker in docker and now build all my images with them!

felbane@lemmy.world on 17 Dec 05:17 collapse

please share, I’m interested in doing the same

scrubbles@poptalk.scrubbles.tech on 17 Dec 07:42 collapse

Sure! I use Kaniko (Although I see now that it’s not maintained anymore). I’ll probably pull the image in locally to protect it…

Kaniko does the Docker in Docker, and I found an action that I use, but it looks like that was taken down… Luckily I archived it! Make an action in Forgejo (I have an infrastructure group that I add public repos to for actions. So this one is called action-koniko-build and all it has is this action.yml file in it:

name: Kaniko
description: Build a container image using Kaniko
inputs:
  Dockerfile:
    description: The Dockerfile to pass to Kaniko
    required: true
  image:
    description: Name and tag under which to upload the image
    required: true
  registry:
    description: Domain of the registry. Should be the same as the first path component of the tag.
    required: true
  username:
    description: Username for the container registry
    required: true
  password:
    description: Password for the container registry
    required: true
  context:
    description: Workspace for the build
    required: true
runs:
  using: docker
  image: docker://gcr.io/kaniko-project/executor:debug
  entrypoint: /bin/sh
  args:
    - -c
    - |
      mkdir -p /kaniko/.docker
      echo '{"auths":{"${{ inputs.registry }}":{"auth":"'$(pr
felbane@lemmy.world on 17 Dec 08:49 collapse

Thanks for the write-up! I’ve been trying and failing to do DOOD and POOP runners via forgejo, but I haven’t had the time or energy to really dig in and figure out the issue. At this point I just want something to work so I’ll give your setup a try 😎

scrubbles@poptalk.scrubbles.tech on 17 Dec 09:04 collapse

Of course! Let me know how you run your containers and I may be able to help on that side too

corsicanguppy@lemmy.ca on 16 Dec 13:15 collapse

It’s still yaml shit though.

Jayjader@jlai.lu on 16 Dec 14:27 next collapse

I dislike yaml as much as the next person, but you can always “just” write Jason JSON (lol autocorrect). Unless I’m misunderstanding your criticism?

chilicheeselies@lemmy.world on 16 Dec 20:52 collapse

Yaml is vette than json for this IMO brcausebyou can write comments in yaml, and in general format multiline strings easier. Json is best for system to system comms. Human to system literlaly anything other text formst than json.

prettybunnys@piefed.social on 16 Dec 14:27 next collapse

What issue do you have with using yaml to define a job?

zr0@lemmy.dbzer0.com on 16 Dec 15:35 collapse

Every language, that uses functional white spaces, is absolutely awesome!!

— no one

[deleted] on 16 Dec 11:57 next collapse

.

h54@programming.dev on 16 Dec 11:57 next collapse

IMO, Gitlab CI/CD blows Github out of the water. They’re not even in the same league. I recommend Gitlab + self hosted runners (it’s so easy).

I’ve been using Gitlab for many years and host my own runners as of the past 6 months because I nearly exhausted my monthly free tier runner minutes one month.

xcjs@programming.dev on 16 Dec 13:40 next collapse

I second GitLab CI/CD - it’s a CI/CD system that just makes sense to me. That doesn’t mean it doesn’t have its complexities depending on your needs, but I’ve overall enjoyed my time working with it.

jacksilver@lemmy.world on 16 Dec 19:23 next collapse

I had someone swear to me that Github templating was better, but I’ve only worked with Gitlabs templates. Why do you like Gitlab over Github?

h54@programming.dev on 16 Dec 21:54 collapse

Gitlab CI feels native. Github offers similar functionality but it feels/looks like an afterthought. I think the Gitlab .yaml structure is more intuitive. Also, how the Gitlab UI visually represents a pipeline is mcuh better, IMO. Self hosting runners on my server (Ubuntu) is so easy and free. I hadn’t tried it with Github but it sounds like it still costs money?!

Note: I don’t work for Gitlab

douglasg14b@lemmy.world on 17 Dec 01:05 collapse

Edit: I forgot this was self-hosted community, disregard.


How does organization work out?

We have dozens of workflows for our monorepo CI/CD stuff. GitHub organization with the flat structure is incredibly annoying.

GitLab is a single file?? (Or am I misinformed? )How does that work out?

h54@programming.dev on 17 Dec 06:36 collapse

The repo specific config is a single file. You can also import templates/other files if need be. I worked in a shop where Devops set up a bunch of templates for generic, common jobs which made getting started easy. If custom config/code is required, overriding a templated job was easy. I was responsible for migrating my team’s ~50 repos (services, libraries, etc) from Jenkins + Bitbucket into Gitlab and found it to be pretty straightforward.

HelloRoot@lemy.lol on 16 Dec 12:10 next collapse

sourcehut.org

iatenine@piefed.social on 16 Dec 12:21 next collapse

Are you sure it was a price “increase”?

I got a similar email this morning but it was the exact opposite of what I expected upon closer examination:

https://docs.github.com/en/billing/reference/actions-runner-pricing

chrash0@lemmy.world on 16 Dec 12:40 collapse

you’re right. i just expected it to be an increase 😅

avidamoeba@lemmy.ca on 16 Dec 13:02 next collapse

Jenkins

prettybunnys@piefed.social on 16 Dec 14:30 next collapse

Jenkins is good enough to be widely used enough to be hated enough to be downvoted.

The sign of a mature product IMO.

You could do worse than Jenkins

avidamoeba@lemmy.ca on 16 Dec 15:25 next collapse

Been using Jenkins since before it was called Jenkins. It’s been in use at every corpo I’ve worked for. It can practically do anything. Especially coupled with Docker.

elephantium@lemmy.world on 16 Dec 21:25 collapse

Hudson? Man, that’s a blast from the past.

synae@lemmy.dbzer0.com on 16 Dec 22:45 next collapse

“It’s the worst one, except for all the others”

PushButton@lemmy.world on 17 Dec 09:19 collapse

I was scrolling, looking for a Jenkins somewhere, to finally find that post with down votes.

Every company I go, it’s a different CICD, and they all make me wish to use Jenkins instead.

prettybunnys@piefed.social on 17 Dec 10:12 collapse

Jenkins is better than many but IMO Gitlab pipelines are top tier.

AA5B@lemmy.world on 16 Dec 17:13 collapse

I’m not entirely sure why all the hate : Jenkins can do the most things the must ways. And yes, it’s so much nicer defining a pipeline with a fully functional language than an assortment of yaml files

Actually that was my response when my company wanted to start using Gitlab ci. It only has one way of doing things so you can probably get a faster start if you had no ci, were a small company, and had simple builds. However we’re over 4,000 builds in many languages from 12 year old monoliths to modern micro services and containers…… and way too much godawful JavaScript. Do you want the quick and simple tool great for a small startup or the all powerful kitchen sink of tools?

cosmicrose@lemmy.blahaj.zone on 16 Dec 13:05 next collapse

I self-host woodpecker-ci.org and I love it. It was easy to set up, and I never have to worry about CI/CD minutes.

EarMaster@lemmy.world on 16 Dec 13:24 next collapse

Gitlab CI/CD pipelines are my go-to tool. At work we self host an instance, for personal projects I use gitlab.com.

corsicanguppy@lemmy.ca on 16 Dec 13:36 next collapse

Where’s the Bazel people at?

_stranger_@lemmy.world on 16 Dec 13:41 next collapse

Magnetic needle. Steady hand.

TechieDamien@lemmy.ml on 16 Dec 14:39 collapse

Not butterflies?

killabeezio@lemmy.zip on 16 Dec 16:24 next collapse

So many these days. Actions are probably one of the best, but there are still plenty of others out there.

  • gitlab
  • dagger
  • concourseci
  • tekton
  • Spinnaker
  • harness
  • argo
  • flux
  • gocd

If I were to pick one, it would probably be dagger. Or really anything but Jenkins.

partofthevoice@lemmy.zip on 16 Dec 21:53 next collapse

Gitea Actions, as well.

chrash0@lemmy.world on 17 Dec 03:01 collapse

we use Jenkins + a bespoke wrapper at work. thats left a bad taste in my mouth enough to avoid Jenkins altogether

Routhinator@startrek.website on 16 Dec 18:35 next collapse

Forgejo and self hosted action workers.

weastie@lemmy.world on 16 Dec 19:36 next collapse

I do devops at work and my experience is that really any CI/CD system works, they all have enough features to do what you want. They all fundamentally just run scripts on boxes. Therefore, I say pick the easiest one, likely the one that is built into whatever Git system you are using.

Try to keep your pipelines simple-ish when you can, they almost never need to be that complicated. 95% of the time it’s just running a command or two. If a pipeline needs to do something complex, I’d recommend writing that script into the Git repo and calling it, rather than having a CI job that is 100 lines long.

chrash0@lemmy.world on 17 Dec 02:59 collapse

this is my experience as well. we have a bespoke wrapper around Jenkins, and the more we can test locally the less time we have to spend waiting for the system to fail. it’s one of the reasons i’ve adopted just to script things locally as if it was CI.

thesmokingman@programming.dev on 16 Dec 19:45 next collapse

Please don’t take me as a GH shill because I’m not. I’m not sure we read the same email given your projects. Actions on GH runners are dropping in cost and there’s a new fractional cost for self-hosted. For the average user, especially those on GH runners, costs are going down. Looking at your repo, you haven’t run anything since July. Your workflow files use GH runners. Nothing in your history suggests you’re leaving the free tier so I don’t get this FUD at all. General Microsoft hate? Fuck yeah. Shitty GH service? Fuck yeah. Plenty of reasons to dunk but this was not one of them. M

elephantium@lemmy.world on 16 Dec 21:35 next collapse

We use Azure Devops at my current gig. It works pretty well for our setup. I’ve used GHA before; it definitely didn’t “spark joy”. I wastedspent way too many hours in the “update yaml file, commit, push, wait 5 minutes for it to fail again” spiral of despairfeedback loop.

Nice thing with ADO is its release dashboard – you get a really nice summary of recent builds and where they went:

$project - dev - test - prod

I didn’t see anything similar for GHA.

douglasg14b@lemmy.world on 17 Dec 01:00 collapse

A lot of that pain can be reduced by writing and running your code locally before pushing it to a CI environment. Generally with our automation we write a CLI, And GitHub actions is just an execution environment that calls the CLI.

And if what you’re trying to do must execute inside an action. You can run workflows locally with docker!

elephantium@lemmy.world on 17 Dec 05:48 collapse

That’s a great idea if it’s possible, but I want to say it wouldn’t have helped with our environment at the time.

I almost wish I could look back at that repo and share the yaml file here, maybe I was missing something back then. I’m certainly more proficient with yaml now.

I do recall wishing there was a way to simulate the execution locally. I think I remember hearing about a local runner, but it had too many caveats to help.

fruitycoder@sh.itjust.works on 16 Dec 23:09 next collapse

Git lab CI is my goto for git repo based things (unit tests, integration tests, etc). Fleet through Rancher for real deployments (manages and maintains state because kubernetes). Tekton is my in between catchall.

verstra@programming.dev on 16 Dec 23:25 next collapse

I’m currently looking into Concourse.

It does have steeper-than-average learning curve, but I really like that it has well-defined fundamentals (resources, jobs, tasks) and isolation with OCI containers. Before I adopt it fully, I want it to run my nix flake dev shell.

douglasg14b@lemmy.world on 17 Dec 00:59 next collapse

GitHub Actions mostly.

The rest is usually plumbing and code to support it. The actions are just the automated execution environment.

trougnouf@lemmy.world on 17 Dec 02:03 next collapse

Self-hosted Forgejo Actions on a Codeberg repository. It was relatively easy to setup and I don’t even need a VPS through my dynamic IP 5G connexion. See also: codeberg.org/trougnouf/cfait

victorz@lemmy.world on 17 Dec 08:52 collapse

connexion

I’m imagining you saying “connex-yun”, and it reminds me of Stewie saying “cool-hhhwip”.

possiblylinux127@lemmy.zip on 17 Dec 05:55 next collapse

Woodpecker CI

TheHolm@aussie.zone on 20 Dec 01:09 collapse

Woodpecker. No BS CI which can be attached to pretty much anything. It just need a webhook and way to pull your project.

chrash0@lemmy.world on 20 Dec 07:55 collapse

nice. simple and modular i like. i deal with far too many “one stop shops” at work to bring that home