Remove more stuff from the build script

2 jobs for main in 32 seconds (queued for 3 seconds)