It's a really interesting world. You can spam GPT to get novel math results but here I am trying to scroll up to the beginning of the conversation and 5 minutes in I still don't know if I'm near the top yet.
Scroll... wait for render... scroll... wait for render... repeat...
We live in a world where there's so much crazy technology but few people use it to make products better or to improve people's lives. Most people use it to just make more money. It's funny too, because there's a million things we could use that tech for that actually reduce costs. Hell, what would be the economic impact of putting ML systems into streetlights so they properly coordinate. Don't even need LLMs for that, and I'm sure it'd save billions of dollars a year. Just a lack of will. I wonder if this will ever change. Is this how we create the high tech low life future?
(FWIW, no problems if I jump into the app. It's purely a web thing, but my point more illustrative than specific)
On my mac if you scroll slightly a blob appears on a slider on the right and you slide it to the top to go to the top. I'm not sure about these hidden user interface features that you only find through trial and error.
In an ideal world, that would be true, but the actual world is very far from that. As an extreme example, AI is very useful to automate scamming people.
Like it was so correlated back in 1600? Under the Romans? But perhaps you just mean that the increased financialization of the US economy means that the percentage of economic wealth devoted to actions without value is increasing? Sort of a modern trajectory to 1789 with the foppish nobles not realizing how much everyone else can get along without them?
And yet you still want to be paid in your currency and use it to purchase the goods and services that give you the best bang for your buck at any given moment. Funny how that goes.
The architecture of our modern technological society lies on top of pure maths. One cannot say “resolving the physicality of the Navier-Stokes equation” will lead to x, y or Zed human improvements, but for hundreds of years it has. It is like Bach making his fugues for beauty and pleasure but occasionally you get chip technology or mathematical ecology or GPS falling out of it.
It always is about money. As long as there's money it will always be about it. LLMs are big techs attempt of shifting money from the people further up, by getting rid of jobs. We're all not angry enough because some of yall are so delusional about the whole thing, by thinking we will get some kind of Utopia.
> What's fascinating is that Google gets virtually zero recognition for this, and beyond that, Google is one of the most hated corporations for it.
The hate it's about what they did with the power they gained.
You can both love and hate something at the same time. Humans have complexity and nuance, don't dismiss that.
It's also worth noting that complaints are more vocal than complements. If things are working well then it's usually quiet. Nothing to say. But when shit goes wrong people talk because they want it fixed. Don't let this bias fool you into thinking that appreciation isn't happening. Appreciation is generally silent while hatred is vocal. That's not isolated to the internet either
It's less a statement about the absolute hatred that Google gets, and more about Google using collected data to target you with ads is the most hated thing about Google (directly or indirectly).
Google pretty much figure out how to take an attribute that every human has (desire) and turn that into indefinite sustainable (and growing!) services for every living human no money required. Role the clock back 25 years and this idea is so powerful and utopian, no wonder Googlers were riding on top of the world. Anyone on Earth of any background and any wealth level will be able to have web search, e-mail, an office suite, maps, youtube, a blog, data storage, all the tools you need for productive online life, for free, for everyone. I cannot think of any business idea more progressive than that.
However, the point is that the above is inseparable from Google's data collection. There is no other solution to achieving the above. None. It's either data collection or pay walls. Walls that naturally run along the huge wealth divides globally, and leave behind those who need it most.
So I suppose at heart the question is: If free services for all humans and mass scale data collection are two sides of the same coin, is this something that we want or something we need to avoid?
(I know people are itching to call false-dichotomy, but lets be real, nothing in 25 years that isn't money-based or ad-based has gained meaningful VC money i.e. VCs dont buy into "donation based" models)
A one point a very hated thing was how they defrauded advertisers using their services. They have increased the fraud over time, so that hate hasn't gone away.
> I know people are itching to call false-dichotomy, but lets be real, nothing in 25 years that isn't money-based or ad-based has gained meaningful VC money i.e. VCs dont buy into "donation based" models
There's plenty of counter examples, from Linux to ffmpeg. You're not wrong about VCs but the argument shifts to if that's the structure we should be using if it's just going to optimize for greed.
But I think you also glossed over the enshitification element. That greed optimization has shifted to become user hostile. There was a time when we could make great products and make good money at the same time. But we've moved to living in a world with oligarchs. They're pushed to make more money than they could ever possibly use (I don't mean need, I mean they couldn't spend it if they tried) because others get to ride their coat tails.
The economy was never a zero sum game. It's possible for everyone to get rich at the same time. You don't have to shit on the user to make a profit. I'll say that if a company is hostile to its users then that company is evil. Most companies, including Google, don't start that way. But being good doesn't mean you won't become evil tomorrow. And Google? They've abandoned nearly all the principles that they used to be praised for. I don't think the criticism is harsh. People are allowed to change their opinions about a company as that company changes its actions
Let's say you could calculate a good-evil score for every large corporation by netting the good they do against the evil they do.
I don't know if Google would be net-good or net-evil, but I'm pretty sure they would be far less evil than most of the other large corporations on that list.
Certainly less evil than Exxon Mobil, Microsoft, Saudi Aramco, Meta, JPMorgan Chase, UnitedHealth, Coca-Cola, Oracle, Palantir, Goldman Sachs, LVMH, McDonalds, etc.
But it's a paradox that on HN, Google gets far more hate than any of these.
Maybe HN is a weird combination of fatalistic and idealistic, where we assume any for-profit corporation with a do-gooder mission is lying about its mission to hide its essential evilness, while secretly hoping that maybe it really is possible to do good and do well at the same time.
For a while it seemed like Google's do-gooder mission was genuine. Unlike their less principled competition, Google refused to take money to influence search rankings. Google withdrew from mainland China rather than continue censoring search results there.
We secretly started to believe. But all the other compromises along the way felt like betrayals.
Google really is less evil, even today, but we hate them more because we dared to believe, and Google let us down.
I think the premise is wrong: I don't think Google is more hated than all of the organisations you list.
Within the tech world, I believe people generally dislike meta, Palantir, Oracle, and Microsoft more than Google.
Out in the real world, in the US at least I'd bet money that UnitedHealth must be more hated. And anyone with at least a modicum of rationality would probably dislike Big Oil and Big Junk Food more too?
---
That said, I think this is an excellent cautionary lesson:
> We secretly started to believe... ...we hate them more because we dared to believe, and Google let us down.
McDonalds should start allowing full 50% off order discounts in exchange for customers opting-in to receiving custom-printed wrappers with hyperpersonalized advertising printed on-demand just for them.
Could they afford to do it? IDK, advertising is effective and seems like a big business.
I think with their McDonalds App discounts for more normal pricing (last I checked), they're half way there?
My Facebook account with no connections was created exclusively to log in for the 20% discount. Facebook is probably getting more out of this arrangement than McDonalds.
I do like that McDonalds employees call me Valued Customer instead of the Facebook Name, after enabling one of the app privacy settings.
For example, Oracle does evil to its customers and partners and competitors. But Google does evil to every human who uses their products. In my book that’s much worse.
> But it's a paradox that on HN, Google gets far more hate than any of these.
There is no paradox. This is Hacker News. It's a site of by and for people in the tech industry. Of course Google's activities (good and bad) get far more attention than those of Chase, or Coca-Cola.
I certainly wouldn't agree that Google "gets far more hate" on HN than Microsoft, Meta, Oracle, or Palantir.
Google can do and has done a lot more harm than McDonalds or Coca-Cola.
They destroyed the open web (over and over - AMP, Manifest v2, etc. etc.), helped turned mobile computing into a rental / second-class citizen model, are ushering in identity tracking and attestation.
They pilfer the commons, unfairly monopolize, expand vertically and horizontally without bounds, make it impossible to compete, tax distribution every single way since they're the only set of rails, profit off of other people's copyrights and trademarks.
They even tried to hoard AI innovation, and it took employees leaving to set it free.
> The hate it's about what they did with the power they gained.
Which is???
Serve ads without selling data?
I'm not sure if y'all are aware, but CC companies sell your data. Your telecom provider sells your data. Banks sell your data.
Google figured out a system to sell access while keeping your privacy. And somehow they're more hated than people who just straight up give your data away.
If most people hate Google all this proves is that most people are retarded.
I share your sentiments re google exactly. But the credit card and banks point is also understated.
Google innovates. CCs are basically a monopoly/duopoly, a private tax on an economy where increasingly everyone uses credit for everything. They make money on these transaction costs and banks issuing them make money charging insane interest when your neighbors fall behind, trapping them in a debt cycle. And yes, your purchases get aggregated into data sales.
Have credit cards innovated? Has your bank innovated? At least with Google their services are largely good and they actually innovate at shocking scale. Especially if you are paying for their products it is hard to argue Drive or YouTube haven't gotten better. You're complaining about ads but you want YouTube to be free? Who is going to pay for it?
Meanwhile has your credit card gotten better? Why is the CEO of MasterCard making millions of dollars? If you had to pick the thing in your pocket that is preying on you and you pick your phone, you're not thinking clearly. The answer is your credit card.
> CC companies sell your data. Your telecom provider sells your data. Banks sell your data.
You're over simplifying. WHAT data being sold is a critical variable. Same with HOW MUCH. Not all data is created equally. Last I checked, my bank doesn't have real time access to my location. My bank doesn't know everyone I communicate with. Not even my telecom operator knows that! None of your examples know every person that is within ~100 ft of me at any given time.
Look, I don't like that the banks and telecoms have as much data as they do, but let's be serious here, Google has some very serious data on people. Don't be so glib.
> Which is???
And don't get me started on all the other things. What they've done to the open web. Email. Smartphones. And much more. I don't know how you can think Google is just an ad company when I'm pretty sure you use so much more from them.
But what is the alternative, and why is it better?
An Apple-esque internet that gets you a full google-like suite of services, along with news and other general entertainment, all ad/tracking free, but for a cool $300/mo? User friendly, pro-consumer, but only for those who can afford it?
I genuinely struggle to say that a user friendly internet that cuts out 80% of the current global internet user base because they are too poor is a better system.
> But what is the alternative, and why is it better?
Is this a legitimate question? There's hundreds of projects working on alternatives[0]. There's also... you know... the internet before Google took over. So I'm not quite sure what you're actually asking here. Is this some kinda "gotcha"?
> An Apple-esque internet
Let me stop you right there. No one wants that either. We want to *go back to* a free and open internet. I don't mean free like you don't pay your ISP[1], but as in you pay your ISP for access to their pipelines and then that's it. That's the contract. No snooping. None of that. They laid the fiber and they're paid for that service.
> all ad/tracking free, but for a cool $300/mo? User friendly, pro-consumer, but only for those who can afford it?
Oh man, the overton window has shifted real far here. I'm not wholeheartedly against ads existing, but I am against Surveillance Capitalism. We can't even begin to have a meaningful conversation if we conflate these things. If "having ads" is equivalent to Surveillance Capitalism, then there's no conversation to be had. Your account is young, so I'm guessing you haven't been around these parts long either. It hasn't always been this way.
> I genuinely struggle to say that a user friendly internet that cuts out 80% of the current global internet user base because they are too poor is a better system.
This is a false dichotomy. It is a lie big tech sells to make you accept what they do. It's no different than the personal carbon footprint campaign[2].
[0] Network effect has multiple meanings here and it often does hinder their goals.
[1] Well it's not like we can truly setup global mesh networks, but hey, maybe one day.
[2] https://www.theguardian.com/commentisfree/2021/aug/23/big-oil-coined-carbon-footprints-to-blame-us-for-their-greed-keep-them-on-the-hook
I've been on the internet since the mid 90's, and thankfully grown up since then. People don't want to work for free, and the people who are paid to work produce far better products for far wider audiences. This is just flatly true
So having established that, repeatedly, over 30 years, it's clear that a functioning internet needs money (what a surprise) to run optimally.
You hand wave away my first point and didn't provide any examples. So please, give me a service or product with mass-market penetration (read: you can frequently find people on the street using it) that isn't ad-funded or direct payment funded. It's hard to come up with one, much less five, much less the tens of thousands that would need to exist.
Your telecom company can triangulate you through their towers, but they don't have access to your phone's WiFi antenna, bluetooth, NFC, nor GPS. Google and Apple have all of that. Snoop on what they're sniffing and I doubt you'll agree that the telecom companies comes anywhere near to what Google and Apple have.
If it wasn't for Google (and ads), my bank might not have even considered selling my data.. They only sold the credit bureaus in the past, AFAIK. Now it's considered a "must have"..
While I agree with you, a large number of people use the term to mean that something was engineered to a complexity they have difficulty understanding. Abstraction is also treated as a dirty word, with people focusing on only one interpretation of the word.
While I don't actually agree with OP, I do agree with their sentiment. I've seen people say something is "over engineered" when there's an elegant design. Elegance isn't over engineering, it is solving problems effectively. It's something we should chase! Elegance is solving the right problem, which usually people are having a hard time seeing. (It's not always easy)
If we constantly let people drag quality down then we get into this frustrating world where everything is constantly half broken.
Anyone saying "don't make perfection the enemy of good" is using a thought terminating cliché, avoiding the conversation of what is good enough. Worse, it is often used by people to drag quality down. If you're creating the "minimum viable product" you usually create a product that isn't working.
But, perfection doesn't exist. Most solutions in the world have no global optima. There are always tradeoffs. You must choose. You must argue with your peers to figure out that tradeoff. Perfection has infinite depth in detail. You must optimize. You just learn the unknown unknowns in an every moving landscape.
But you should still chase perfection. Like you chase a utopia. Like you chase your dreams. There's always something to improve on. Chasing perfection while knowing it can't exist means you will continue to search for the flaws. It means you will continue to improve. "Over engineer", because that's just engineering. Make things actually work, while recognizing they're always broken somewhere. Don't get offended when someone points out a flaw, you already know it's not perfect, so figure out if it is a tradeoff or can be fixed. Just keep improving things, because otherwise they keep getting worse
> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked.
How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the next easiest things. They're "the easy things" from that standpoint but that doesn't mean they aren't harder than "the easy things" when you started. Complexity increases as precision increases.
The goal of a modeling exercise like this, which you don’t have to buy, is to generate a simple set of initial conditions that can explain things we already know. Then, we can manipulate some initial parameter value to make predictions about things we don’t see, but might.
Likewise, it is obvious that gravity exists, but a simple model that explains where it comes from (in quantum terms) would be a big breakthrough iff it came with plausibly testable implications that could be tested via experiment.
In a weak sense, singularities are common and should be expected every so often. In a mathematical sense, a singularity is where a model 'blows up' and neglected terms become part of the dominant balance.
Industrialization hit a point of diminishing returns, but the industrial revolution was nonetheless a 'singularity,' where life afterwards was qualitatively unpredictable to people who lived before. Likewise, agriculture was such a technological singularity to hunter-gatherer ancestors.
I could even make a decent argument that writing and literacy were such a singularity, making inconceivable social organizations routine.
In that weak sense I expect AI to be a singularity, recursive self improvement or no. Life in 2050 may be completely unpredictable to someone who was taken out of time in the year 2000.
The remaining questions are speed and intensity, and both of these questions are related to RSI. If RSI works, then the 'fast takeoff' visions become more plausible where society transforms over months to a few years – at least locally where the enabling technologies have diffused. If not, it might take a couple of decades.
People have other dreams, and have written about them in science fiction. But science fiction can just handwave away resource problems that exist in real life so that the plot can happen.
Just as the popular "grey goo" nanomachine disaster can easily be shown to be impossible due to resource imbalances and energy shortages, AI recursive self improvements rapidly slows down due to problems with complexity, training data and compute availability (whether due to actual processors or just due to energy demands).
whether or not the singularity would be best thought of as science fiction, it was not presented as such, and has not been taken as such by a great number of people, and has formed the basis of a lot of opinions as to how things will pan out.
It is these opinions that make the conclusion the parent poster supposed should be obvious to everyone not obvious to so many.
Because they depend on whether the rate of improvement of self-improvement outpaces the rate of increase in difficulty or not, and at some points they clearly do - e.g. a lot of skills makes the relative rate of subsequent improvement easier for a while.
It may seem obvious that it can't last, but showing the conditions where it can't still matters.
To put it another way, the cost per advancement increases. That’s “as we advance it becomes more difficult to advance”. However, because of the prior advances, you also have more resources to throw at future advancements.
So, then, the question is whether the “profits” on the last advance are enough to pay for the next one. We can define a new term, “affordability”, as what % “profit” you can expect from each advance relative to its cost, telling us whether it becomes relatively easier or harder to continue to advance.
You could argue that there is a capital built up - but even that is difficult as a lot of the knowledge for, eg., building LLMs can not demand rent. Everybody can build their own transformer networks.
> To put it another way, the cost per advancement increases.
This is not true when you don't use capital that demands rent. On the contrary, the cost per advancement is actually decreasing as we develop more knowledge.
LLMs are a cognitive technology (as contrasted to a communication technology). And it will help us tremendously manage knowledge such that you can utilize it better decreasing the cost of advancement.
Whether or not it decreases the costs of advancedment would depend on what is causing advancement to have costs. Some things might be addressable by cognitive technologies, others might not.
If capital doesn't create a return evenutally, why would it invest? (Or did you mean "rent" in the narrow economics definition? But then not sure how that applies here).
> We have advanced tremendously over the past 200 years
Would most of that have happened if we hadn't found oil, though?
It's difficult to precisely identify how much of progress is owed to intelligence, and how much is simply energy availability. Energy is intrinsically transformative -- the dumbest of organisms can take over the world in a blitz if they can process available energy better than the others, whereas the smartest of entities is going to be ineffective without fuel. Intelligence can, of course, unlock the capability to exploit new energy sources, but there's still an intrinsic physical distribution of it that's outside of anyone's control.
Maybe, maybe not. The point is that you can't simply will energy into existence. Its availability, including the ease of exploitation, is a physical given. Which is true of resources in general. Arbitrarily advanced technology can't simply make exploitation arbitrarily easy, so for example there can always be situations where the very best of all possible technologies will be less productive than setting fire to petrol would have been, if there had been any. It's complicated. It's not just what we do, it's not just what we have, it's both.
Probably the best test of how the future might trend long term is to see how well we can extract value from all the stuff and waste we are currently producing, because these will inevitably become the primary resources from which we have to build everything new. Unfortunately that shift is likely to happen extremely quickly and catastrophically, because typically you only start processing bad materials once you're done with the good.
Then use batteries as an example. Going on 200+ years now most of the major changes have been an easy swap: chemistry. Lead acid --> nickel cadium --> nickel metal hydride --> lithium ion. Improvements in batteries are hard and the surrounding ecosystem has gotten better. Better BMS and better motors have also made gains to increase the efficiency of the battery. Compounding gains within the ecosystem doesn't truly improve the battery directly, but improves them for implementation and use cases.
All the remaining problems with batteries are a hard solve because it's tradeoff after tradeoff. Lithium sulfur, for example, could be amazing but die after a couple hundred cycles at most [0]. So... If LLMs are so novel, as many continue to claim, why hasn't this been solved? Probably because LLMs are an interpretation of our current understanding of everything. An LLM currently only does what any human can do, albeit in a compressed timeframe, mostly.
I think it follows. LLMs have made some considerable improvements, but use "reasoning" (really just a mechanical, autoregressive, loop: generate tokens --> append tokens to context --> feed the expanded context back into the model --> generate the next token --> repeat) as an example. You can do this by hand as well in a traditional chat volley, but it's slower. So we improved LLMs by putting them in an automatic loop (oversimplification - but at the end of the day holds water).
Until LLMs showcase true external breakthroughs that aren't driven by human guidance (not happening anytime soon) we are in this loop of hacks being used to improve the 80% (the LLM itself). Notable jumps? "Reasoning" and with Mythos-like models we now have "Advanced Reasoning" (by leveraging a more capable looping framework such as an agentic harness).
It seems the rate of advancement has increased over time though. Like from ancient times to 1800s the rate of progress was slow, then basically a relative explosion of progress compressed in 200 years.
> It seems the rate of advancement has increased over time though.
Effort and expenditures are not fixed. You cannot hand wave away these variables.
Here's an example let's compare two investment strategies:
- Alice puts $1 into VOO every day.
- Bob puts in $10000 into VOO every day.
You don't get to say "wow, Bob is a much better investor than Alice. He's so smart, look how fast his money grows!" You get to say "Bob is investing more money than Alice."
Now pretend Bob is investing into a slower growing account than Alice. His account will still grow faster as long as his account is not losing >99.9% per day. He could have a worse strategy and still grow faster because he's throwing more money at it
Because of access to a giant store of energy. Our advancement has also created a future need to consume more and more energy to fuel increased complexity.
That RSI can be bottlenecked? I guess this is obvious to many people. Whether RSI will be bottlenecked (at some not very interesting stage) is another question.
Our intelligence was enough to turn some balding apes into atom-bomb wielders and astronauts over a few thousand generations.
Consider a hypothetical: it is possible to make a semiconductor clone of a human brain, that runs at semiconductor speeds, has semiconductor size-scale, and has a power requirement of the Landauer limit.
This copy would be smaller than a pea, and you'd get to pick on a sliding scale between "same power draw as human but thinks faster than us to the same multiplier that we jog faster than continental drift" or "thinks as fast as we do while using around a few µW of power".
We do not know how to do this. We don't know how far we are from figuring out how to do this. We do know* evolution made our brains, and we do use simulated evolution as a standard technique in machine learning.
But it may well be that just as no human knows how to design a human mind, we find the best AI we can make don't know how to make a better AI, at which point they're stumbling blind in the dark: while evolution is a neat method, it is limited, blind to what the best next step is at any moment.
* well, those of us who are not creationists, at least.
Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. If the "pea-sized semiconductor brain" is not a surprisingly shallow problem that you can just solve by interpolating existing research, I don't really see the LLM-approach to AI being the thing that makes something like it happen.
>we do use simulated evolution as a standard technique in machine learning.
Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications.
> Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability.
These are not the only currently-in-use AI models. However, recent news has even this particular category of AI solving multiple previously unsolved Erdős problems.
(Even if they also do stupid things on a frequent basis).
> Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications.
I said "standard technique" rather than "best" for a reason ;)
This is proof-of-possibility: natural selection did it, we know how to mimic that, but we don't know enough to be sure we're doing it with a reward function that will actually give us minds like ours on an interesting timescale with a probability high enough to care about.
Machines have solved tons of unsolved problems in mathematics. That's not a proof of intelligence.
If you brute force a solution, we congratulate you on your effort.
If you stumble into a solution, we congratulate you for being lucky (if we can distinguish)
If you find a unique solution that no one else imagined, we congratulate you on your intelligence.
These are categorically different things and the difference matters. It's the whole distinction. Though in the real world success usually requires all three (and more), complicating evaluation.
When we're talking about intelligence you can't distill it to "getting the answer". If you do then I'll direct you at an abacus, a calculator, a watch, or google search if you want to look at super intelligence.
> natural selection did it
Did what? Gradient descent? If that's the argument, you need to read more
> If you find a unique solution that no one else imagined, we congratulate you on your intelligence.
If that's your definition, bad luck: that is exactly what the AI did.
There are other definitions where AI fail, my example of which would be "how many examples did it take to learn the basics?", ML is as thick as plankton by this definition.
> When we're talking about intelligence you can't distill it to "getting the answer". If you do then I'll direct you at an abacus, a calculator, a watch, or google search if you want to look at super intelligence.
At which point I direct you to the words "combinatorial explosion", how the busy beaver function grows faster than any computable sequence, and how many symbols exist in mathematics.
Valid number of Chess games? Estimated to be between 10^120 and 10^123
Valid number of Go games? Estimated to be between 10^10^108 and 10^10^171
Busy beaver number corresponding to something connected to the Collatz conjecture? BB(15). Nobody knows the exact value of any BB number bigger than BB(5), though we do know the lower bound for BB(7) is > 2↑¹¹↑¹¹3; needing to use up arrow notation is always a sign things are getting wild.
So far as I know, nobody's even bothered to work out what BB number would correspond to the recently solved Erdős problems. But we can look at the size of the lean proofs used for them and say "lol no" to the idea of brute-forcing it inside this universe before heat death gives every particle and photon a wavelength larger than the cosmological horizon.
> Did what? Gradient descent? If that's the argument, you need to read more
If you were to go up the conversation tree two more steps, you would already have the answer:
Me: We do know* evolution made our brains, and we do use simulated evolution as a standard technique in machine learning.
defmacr0, replying to "we do use simulated evolution as a standard technique in machine learning.": Well, not really. [says it's limited with examples] in practice and I'm not aware of any notable applications.
Me: This is proof-of-possibility: natural selection did it
Natural selection made us. That's the answer to "Did what?"
...during autoregressive pretraining ([2]). A model pretrained on texts with 1930 data cutoff can solve a few programming problems when given a few examples. Its success rate is understandably much worse.
Remember, I was saying "by this definition", where "this" isn't about capabilities, but about effort needed to get there, which in the case of your link is "260B tokens of historical pre-1931 English text".
A human who read 260e9 tokens will take something like 1900 years of 24/7 reading to get that far, while a more realistic human (though one who still reads a lot) would take 260e9/((50000/0.75)*365) ~= 10,685 years, and that still only gets you something "interesting" rather than "competent".
Don't get me wrong, even merely HumanEval pass@100 ~= 0.04 (eyeballing that chart) shows the model has clearly learned something and isn't just randomly throwing things at the wall. All I'm saying is this took a huge effort to get even that far (and, implicitly, that this is a reasonable argument to use if you want to say they're "not intelligent").
Autoregressive pretraining (text/images/video prediction) produces a foundational model. You can look at it as a highly compressed conditional probability distribution of the human brain output. The information-theoretically optimal compression of the data is a program that reproduces functionality of the process that generated the data.
So, it stands to reason (and observations) that such a model captures not only surface statistics of the data, but a part of functionality of the system that generated the data (the human brain for text, physics for video).
Is it possible to clone without destroying the original brain (and is a semiconductor the right substrate for it)? Thinking of the no cloning theorem here...
The no cloning theorem applies to quantum states; when it comes to information processed by it, it's unlikely that a big wet messy hot thing like a cell is usefully treated as "a quantum state", let alone an entire brain.
However, this is irrelevant to the thought experiment, for which it is sufficient to merely be as competent as a human, rather than a completely quantum-perfect replica. Clone in the sense you would use the photoshop brush, or perhaps in the sense you might be sued for trademark infringement.
What is the thought experiment then to show other than a hypothetical piece of compute hardware? What's the experiment and its implication beyond the existence and its properties?
> As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done.
This isn't some foregone conclusion. It completely depends on the rate at which the intelligence and abilities of the AI increases. If that rate was high enough, then the harder and harder problems would become easier and easier for it.
Actually, evolution seems to show the opposite: The rate of advancement has only sped up, with billions of years between significant changes going to millions, to thousands, to tens and arguably to mere years now.
Having said that, we're probably looking at an S-curve with the physical limits of reality getting in the way in the end.
The more you evolve I guess there's more surface area to evolve on. Similarly cavemen could only do so much. But today you can take a degree in physics, art, history, etc. and it's probably impossible for one person to know all that ever was and is. But it's possible for a caveman shaman to know everything the tribe knew up to that point. Probably. So while it's harder to make progress there's also a lot more places you can progress.
You're ignoring a lot of important factors. I'll give you most obvious ones. We often do genetics testing on fruit flies because they reproduce quickly and the initial population is large.
Your claim ignores this fact. The base population and number of interactions has increased.
I am not ignoring anything, I'm looking at the broader picture, which includes non-biological evolution. Simple rebuttal to your specific point: The population of self-improving AIs will also go from 0 to many more.
In a broader sense evolution moved from very static simple domains to dynamic malleable complex domains. Biological evolution speed is glacial compared to cultural evolution speed. Even then, cultural evolution is fairly slow compared to technological evolution.
"How is this not obvious to everyone? As we advance it becomes more difficult to advance. "
It's a type of recent change blindness. Because for the last few years there are seemingly impossible breakthroughs every few months. Literally, things everyone said would take 30 or 100 years, happen within months. It is really easy to think this will continue. The new normal. Really, we haven't seen it slow down at all, so why would we expect the miracles to stop all of a sudden?
It seems like accelerating, and decelerating, are both fair game as future directions at this point.
So it is nice that the paper put some weight behind the argument that this could all grind to a halt for a lot of different reasons beyond the hype.
> Literally, things everyone said would take 30 or 100 years, happen within months.
"Everyone"? Plenty of people in the 50s said we'd have sentient humanoid robots before the year 2000. And flying cars. Sure, many people underestimate the rate of change. But techno-optimists have consistently overestimated it, and they still do.
> I didn't mean to include science fiction authors, or extreme futurists.
Science fiction authors are quite influential, and I don't think they were the only ones either. I expect that plenty of actual scientists had similar views. It wasn't extreme.
> In the 90's. I remember people talking about voice recognition will take 50 years.
Sure. Who? How many people said that? How many people said the opposite? If we're doing anecdotes, I remember that circa 2010 it was a fairly normal belief that fully self-driving cars would be widespread by 2020. I mean, I thought they would.
Point is, many people say many different things, there's never been any widespread agreement about what the future holds. They underestimate, they overestimate, but mostly reality moves sideways in directions nobody thought of and which are somehow simultaneously way more impressive and way less impressive than our expectations.
Technically correct. But kind of backs every argument into a corner. I'm assuming also, anything like a paper with data, can also be questioned in the same way. Don't object, I've seen that done. So basically, don't talk about anything, because anything you say is contradicted by something somewhere. Everything is True, Everything is False, everything is subjective. There is no reality. Every person has there own point of reference that is completely subjective and invalid.
> I'm assuming also, anything like a paper with data, can also be questioned in the same way. Don't object, I've seen that done.
Unless you've seen me do it, I don't see why I can't object. What a strange thing to say.
Look. You've supported your argument with vibes and anecdotes, so I'm offering contradicting vibes and anecdotes of my own. All that means is that it's not as obvious as you thought, and it is worth digging deeper (or not, it's not that important). A paper with data would certainly make a stronger argument. And of course it can be questioned, but it cannot be questioned in the same way. That you've "seen it done" is irrelevant.
Fair but there's two things I think need to be added
1) predictions are usually made assuming either constant effort or continued growth. They will be extremely conservative when the environment changes and you start dumping trillions of dollars into solving those problems.
2) while there's been real progress that doesn't mean it isn't being over sold. We do need to differentiate the signal from the noise
Probably because it's not true. We had shitty neural networks for decades before the recent explosion. That particular branch may be a dead end, but there could be others lurking and waiting for their time.
Because everyone's thinking around intelligence is incredibly muddled by a variety of factors, and no one is particularly motivated to actually correct anyone's mistaken notions on the matter.
I'm conflicted. On one hand I think we should more openly call people idiots and push back. On the other hand there's Descartes argument for idiots in good company.
I just wish all the people that claimed to care only about truth would actually care about truth. Feels like society is more that trope where someone says a joke to a crowd and no one hears it except one charismatic person who repeats it and gets all the laughs. In reality it feels like the repeated version of the joke doesn't even make sense, it is just vibes.
Honestly, if anything the library of congress should be operating a system similar to the way back machine. Isn't preserving historical information one of its objectives? And what libraries do in general?
But I'm very in favor of maintaining "the record", as it were, for government websites. If we can have changelogs on bills then we should elsewhere. It informs the citizens of the actions of our government. What has changed and "who done it". That can go both ways and I hope it would incentivize those trying to actually do good and not just treated as a liability.
Hell, if the NSA can just gobble up all the Internet traffic and store it on servers in Utah then the least we can do is make public records accessible. The archival work has already been done and we've already paid for it
Its called legal or mandatory deposit. In some countries their national library is required to crawl the open internet within their language, besides collecting regular published materials. In the US laws on legal deposit has not been extended to non printed materials.
Every little fight is what got us here, but that's also how we get out. Do good for the sake of good. Don't let others push you beyond your ethical bounds.
The good thing is we don't need everybody to do this. Even a small percentage can build momentum. So speak up and back up those who do. It makes it easier for others and causes people to be more nervous to float unethical ideas.
reply