I did a cursory search on this, and couldn’t find anything, however, is there or could there be an llm trained only on text or data which is in the public domain or is otherwise openly licensed?
Nvidia Nemotron has a published dataset, so if they had copyright data in it it would be very easy to determine.
https://huggingface.co/collections/nvidia/nemotron-pre-training-datasets
👎 https://huggingface.co/datasets/nvidia/Nemotron-CC-Code-v1 They’re using Common Crawl and there’s no mention of filtering for public domain licenses in this text. Pretty safe to say it’s probably a bad dataset. (In my opinion. I’m not a lawyer. This isn’t legal advice.)
If that dataset has copyrighted material, nvidia committed a clear copyright violation by publishing it publically. I’m happy to trust that nvidias lawyers have vetted it properly.
Don’t confuse having deep enough pockets to dissuade lawsuits with actually adhering to the law.
Anthropic has deep pockets, still got fined. Deep pockets also makes you are target.
Do they though? They have investments, but a product that has never made money.
Nvidia has capital.
Deep pockets in that their investors seem happy to pour money into a pit…
Fair enough. But still less liquid than real capital, which is my only point.
Litigation is mostly too expensive for anyone but the super rich, even for more basic things.
I’m happy to trust that nvidias lawyers have vetted it properly.
Yes, but while you would vet it by “what complies with laws”, large companies use the “what can we getaway with” metric. And if the only penalty is a fine, they will just consider that cost of doing business, and do it anyway.
True, but as per the anthropic settlement linked in another comment, the penalties are pretty significant. 1.5B is a pretty reasonable fine against a 59B profit. Copyright still has some pretty significant teeth.
Its not like its antitrust law /j
. 1.5B is a pretty reasonable fine against a 59B profit.
How is a single digit percentage of profit a penalty?
It would need to be >100% to dissuade imho
It could always be higher, but what percentage would you actually be happy with? Most civil fines are well under a percent, they are definitely a penalty.
The way I look at it, nvidias nemotron division isnt bring in anything close to $1.5B by itself. If they get hit with $1.5B fine for copyright infringement, the nemotron division takes a huge loss, which the rest of the company has to pick up the tab for. That isn’t going to make investors happy, and they would be out for blood. It would impact earns and may cause actual damage to their valuation. (Whether its enough is unknown, the market makes no fucking sense at the best of times). If I lost 1% of my companies profit I’d lose my job. Its enough of a deterrent that the people involved should be paying attention.
On the other hand, Anthropic is playing with magical funny money, so $1.5B really doesnt matter at all to them. And they don’t answer to investors, because they think they’re build god…
For a business, something costing them a single digit percentage of profit is just the cost of doing business.
They should lose double digit or more profits as a company plus individuals, starting at the top, should be penalized financially and/or with jail time depending on the offense. Yes, I mean mean some penalties should eat into future profits as well.
You think the wealthiest business on Earth is afraid of a little thing like copyright law?
oh you sweet summer child… (edit: sorry for my attempt at a lighthearted quip that people ended up hating 🫣)
(I’m just linking random lawsuits. I’m not a lawyer, this isn’t legal advice.)
Also, there are a ton of licenses where reproducing them in full is probably using them as intended, think of the GPL for example. But that doesn’t say anything about plagiarizing excerpts without attribution or licensing info. So just putting them on hugging face is potentially not quite the same as basing an LLM on it, I think. Anyway, I’m not a legal person so don’t ask me for the details - but you’ll be able to find plenty of articles on this by people actually qualified to talk about it.
oh you sweet summer child
There is no need to call me a sweet summer child.
I’m not terribly swayed by a Munich regional case, especially as its still under appeal.
The anthropic settlement definitely is a bit more meaningful though, although it does contradict the Munich one:
We reached this settlement in 2025, after the court’s landmark ruling that training AI on books is fair use under copyright law
The basically settled because they stole the copies of books they used to train on. If they had just paid for a copy of the books they possibly would have been fine.
My apologies, it was an attempt at humor which I can see backfired. 😬
Now as for the case selection, I personally find the regional case interesting because it appears to show the possibility of courts seeing this differently. There are more out there than the ones I quoted.
And I think there might be a possible misunderstanding, sorry about my lack of clarity: I wasn’t trying to prove that this is for sure a copyright infringement, since I’m not a lawyer anyway. But I think it’s pretty obvious NVidia doesn’t care if it turns out to be one - that’s at least my personal opinion.
The regional case isnt interesting because its regional. Regional courts get it wrong all the time, only for it to be overturned by higher courts. There is also the possibility that this court was known to be pro-authors or anti-big tech, and the case was brought there for that specific reason.
If it gets held up on appeal then that is much more interesting, but for now, its basically one judges opinion.
In my uninformed opinion, it casts quite some doubt on the narrative that it’s absurd that courts will somehow decide against AI, or that the use will “clearly” not ever turn out to be a fundamental problem later. And again, there are multiple cases where this happened, it’s not just one judge.
But I’m not a legal expert so keep that in mind and don’t trust me. (However, I’m making an educated guess that perhaps you aren’t one either. My apologies if that is incorrect.)
I’m not a lawyer, this isn’t legal advice.
What an annoying and pointless thing to write several times in a row. One time was more than enough.
deleted by creator
Many of these comments are not even construable as legal advice but honestly even if they were that wouldn’t be on you if someone was dumb enough to follow it
They’re also some of the better open weight models
Ive never tried them tbh, have you? Do they work for code?
Not personally, the larger ones rate well, but may be out of reach for most home users due to hardware requirements. Generally open weights models have been about about 3 to 6 months behind the big closed frontier models
Apertus. Or at least so it claims, I haven’t checked if their data sources (listed in this paper) corroborate their claims about them.
👎 Doubt it: https://github.com/swiss-ai/apertus-tech-report/issues/10 (I’m not a lawyer, this isn’t legal advice.)
Wasn’t there one that was trained on pre ww 1 books or something
Probably, the more relevant question is what can such models do.
Translation would be nice.
Also finding bugs. Just training it on all historic CVEs of Foss like WordPress and the Linux kernel would be a huge plus to CI pipelines.
Once there is one, people can test. It would be interesting for sure. (I say that as mostly an LLM hater. But I’d like to see adoption of a solution that is less problematic for everyone. I think most people would agree that would be an improvement.)
