spoiler
When I say LLMs I always mean opensource, openweight models not OpenAI/Anthropic/X-AI crap, these companies are trying their darndest to spread misinformation and lobby to quell and prevent the development of these opensource models, they should be punished by creating an alternate ecosystem untouched by the opinions and whims of these feudalistic tech lords
The exam/criteria should be broad and rigorous, The license issued and the exam should be private(tied to a handle not PI name), unique and publicly verifiable.
Right now the onus is on individual projects to take a stance for/against LLM use for any contributions in their codebase.
With all the opinions floating around regarding LLM use for programming(most of which i feel is either misinformation, or technophobia), I think this would at least stem some of the legitimate concerns regarding competency and responsibility.
What kind of criteria or tests are you imagining?
A certificate after a test would only show capability, not continued practice. So my confidence in it wouldn’t be very high. Demonstrated practice seems more applicable than some cert some time ago. Some people chase certs to check boxes rather than prove or apply the practice.
What kind of criteria or tests are you imagining?
I am imagining a sandbox environment where the test taker is provided with a medium tier capable LLM, and they are given tasks things like
- given a huge codebase, can you implement a new feature following the coding standards of the codebase and minimally changing the existing code
- a job to change something and follow the cascading effects through all the files it touches, fixing it without breaking things elsewhere
- ability to reign in a LLM that is doing more than you asked it to do
I am sure more experienced people would lay out more scenarios to test for
A certificate after a test would only show capability, not continued practice. So my confidence in it wouldn’t be very high. Demonstrated practice seems more applicable than some cert some time ago. Some people chase certs to check boxes rather than prove or apply the practice.
True, but then again a guarantee that the person knows what they are doing is better than no guarantee at all
I always mean opensource, openweight models not OpenAI/Anthropic/X-AI crap
Honest question: is it even possible to separate oneself completely from AI’s multifaceted exploitation by using “FOSS AI”?
And if yes, does such ethically acceptable “FOSS AI” exist?
who is funding this? who gets to decide? the real.issue is, there has always been slop submissions but the barrier of entry wa shigh so few people in civilization had the skill and fewer even attempted to submit. now every si gle person alive can submit. its not an anti llm stance uts “how do i manage this side gig i dont get paid for” stance
who is funding this?
Hopefully some non profit
who gets to decide?
By some new standards community that isn’t biased towards corporate
the real.issue is, there has always been slop submissions but the barrier of entry wa shigh so few people in civilization had the skill and fewer even attempted to submit. now every si gle person alive can submit.
That is what I am hoping my idea would solve
What is the problem that this would solve?
drown out the slop. If a slop kiddie makes a 600+ diff PR, you can safely ignore it, but if a license holder makes a PR you consider it.
how does this compare with something like the vouch system, possibly with some sort of federated mechanism, so you could choose which list of users to trust
It might discourage new contributors and create a sort of chicken and egg situation, also a vouch system has the inherent risks of being camped by bots
I think having to get a cert is even worse.
and some will get camped by bots, but that’s fine since different projects have different tolerances for contribution quality. I’m thinking like the bluesky moderation system, where each repo can subscribe to a particular pool of vouches in addition to their own. you may have lists for no ai, lists for responsible ai/abides by repo rules, or good intentioned but pure ai and minimal human contribution.
How do you identify a “slop kiddie”? It’s not like LLM users identify themselves as such.
You’ll have to judge by their code, at which point you’re already not ignoring them at all.
in my mind a slop kiddie is no different than a script kiddie, both use the tools they have found with no understanding of how it works, both rarely contribute anything meaningfully new.
as such there are tells, of course I am not suggesting we ignore every anonymous PR, but the ones that broadly fit into certain patterns
Great idea. Bureaucracy and reducing human ability to a single score is always the right process for assuring quality.
Hey, if we don’t democratize the ability to get that single score some corporate will start charging money to do it and in a few years that will become the standard
democratize
🤣
Hey man, I understand it doesnt often feel this way, but yes bureaucracy IS an implementation of democracy.
Food standards, water standards, data protection laws, environmental laws, building standards, etc…
are all bureaucracy by the people to protect the people.
If you’re competent, you don’t use LLMs.
So I don’t see the purpose.
Don’t you have something better to do with your life than spam obscure corners of the internet with your aggresively ignorant opinon on this subject? You have no expertise or experience in the matter, why are you so confident in your opinion on it?
Idealists believe that ideas shape reality, so they fake confidence to try convincing others that whatever they want is already real.
According to idealism, the thing they want is then supposed to magically appear when enough people believe it to be real.
And when that inevitably doesn’t work, they blame the people that called them out on their fake claims, for not believing hard enough.
Maybe a year ago. Not today.
