I'm arguing that this is to simple of an explanation.
The claude paper showed, that it has some internal model when answering in different languages.
The process of learning can have effects in it, which is more than statistics. IF the training itself optimizes itself by having a internal model representation, than its no longer just statistics.
It also sounds like that humans are the origin of intelligence, but if humans do the same thing as LLM, and the only difference is, that we do not train LLMs from scratch (letting them discover the world, letting them inventing languages etc. but priming them with our world), than our intelligence was emergent and the LLMs one by proxy.
Since the rise of LLMs, the thought has definitely occurred to me that perhaps our intelligence might also arise from language processing. It might be.
The big difference between us and LLMs, however, is that we grow up in the real world, where some things really are true, and others really are false, and where truths are really useful to convey information, and falsehoods usually aren't (except truths reported to others may be inconvenient and unwelcome, so we learn to recognize that and learn to lie). LLMs, however, know only text. Immense amounts of text, without any way to test or experience whether it's actually true or false, without any access to a real world to relate it to.
It's entirely possible that the only way to produce really human-level intelligent AI with a concept of truth, is to train them while having them grow up in the real world in a robot body over a period of 20 years. And that would really restrict the scalability of AI.
I just realized that kids (and adults) these days grow up more in virtual environments behind screens than in touch with the real world, and maybe that might have an impact on our ability to discern truth from lies. That would certainly explain a lot about the state of our world.
A few years back i saw a documentary about kids in a third world country were it is normal to use plastic bags for drinking soda.
These kids couldn't understand that the plastic garbage in their own nature is not part of nature.
Nonetheless, depending on what rules you mean, there are a lot of people who show that logic or 'truth' is not the same for everyone.
People believing in a god, ghosts, conspiricy theories, flat earth etc.
I'm more curious if the 'self' can only be trained if you have a clear line of control. We learn what the self is because there is a part which we can control and than there is a part which we can't control.
An unknown threat, potentially from the supposed nation-state target itself, has a very high risk.
I'm not versed in creating ultra-sterile lab conditions -- things can escape VMs, escape your network, nothing is impossible. Do I instead bring it to my employers systems and let them take the risk? And to what benefit, when I can just wait?
Fair enough, my morning brain didn't think cloud, though i guess one could argue you're still passing off the risk onto someone else. Either way, its not my expertise
AWS is expensive, in my mind, because of stuff like this. They don't want you to nirror it on aws, so egress is expensive. The $/GB/month storage fees it'll cost to store this while exploring it is not cheap, either. And once you have an idea of the data you want to move out of the gap, you want to process /extract it quickly (because of $/GB/Month costs...)
I just thought about a spare machine I have with a 12TB spindle and an SSD not plugged into a network.
I understand how to airgap, and unless something can magically worm it's way through HDMI that's probably how I'd get data out, just to be annoying to everyone. To be fair.
A EC2 (vm) on aws with a little bit of CPU, Memory and enough storage attached, costs 1k per month which is something like $1.5 per Hour.
Its not necessarily about storing it longerm, its about 'looking into it'.
I don't get the Airgap thing though at all. There is a very minimal chance that this contains a zero day. The idea of a zero day is, that you can attack systems and you sell it to people who have high profile targets or systems.
Some random person downloading leaked data, everyone can download, is not a real target for a zero day.
And a zero day which breaks random unpacking tools and your vm/system, would be worth even more.
> I'm not versed in creating ultra-sterile lab conditions -- things can escape VMs, escape your network, nothing is impossible.
I suppose it is a bit hard to find hardware without integrated wifi these days. Maybe taking a sbc (pi or whatever) and wrapping it in tinfoil would work?
You could always cut the pcb lines if you want that guarantee.
I'm aware I'm being cautious to the point of paranoia, but anything with the Russian gov is just not something I feel like learning about the hard way, even if I think I'm able to make such a safe environment
To me it goes beyond "civil service" and becomes more like "military service" - you're directly putting yourself in harms way for the collective good. It's not reasonable to expect many users on HN have the setup required to investigate this - sure we're all interested in technology. But we're not all cybersecurity experts.
This is the equivalent of your grandma thinking you're a tech genius because you can restart the router. The skills required for this kind of work are specialised.
Why asking people to do something you should have done first?
If there's anything worthy in it, then point to those interesting documents where HN community would be more than happy to help.
The GP sounds like one of these people who describe themselves as self made, or libertarian, where history begins where you like it and coalitions are only worthy when you’re the biggest benefactor. Best to ignore and let the leopards find them.
Cool! We use yolo and have good success after labeling 1k of images but i'm happy to try it out.
Does AGPL mean i can't use my model for my image detection or does it mean i can't use your software if i would want to provide finetuning service (which i don't want to).
Hi Sonnigeszeug, great that you're looking into LightlyTrain!
We designed LightlyTrain specifically for production teams who need a robust, easy-to-use pretraining solution without getting lost in research papers. It builds on learnings from our MIT-licensed research framework, LightlySSL (github.com/lightly-ai/lightly), but is tailored for scalability and ease of integration.
For commercial use where the AGPL terms might not fit your needs, we offer straightforward commercial licenses for LightlyTrain. Happy to chat more if that's relevant for you!
I do assume that the notion is used and also implies the 'resolution'/'precision' of that number.
reply