It would require government permission. The top labs would be agreeing not to compete on model advancements for a period of time, which seems like a pretty clear violation of antitrust law.
How is this a clear violation of antitrust law? And have any of the leaders of these companies said that they are asking for government permission for this reason?
The entire essay is about stopping development until it can be made more safe, with specific ideas on how to do that.
The fact that he would suggest this despite preparing for an IPO is an even stronger signal that it's in good faith -- it will almost certainly delay or reduce the valuation of the IPO.
> The fact that he would suggest this despite preparing for an IPO is an even stronger signal that it's in good faith -- it will almost certainly delay or reduce the valuation of the IPO.
No it won't.
This is a call for a "safety cartel"[1] in which the dominant firms become more entrenched against competition by colluding to limit the progress would-be competitors can make in the market.
> The entire essay is about stopping development until it can be made more safe, with specific ideas on how to do that.
No, the essay does not talk about "stopping development". That's a very clever sleight of hand made in the essay to get readers to draw this false conclusion.
It just talks about building AI at a "balanced rate". Untangling the corporate speak, this amounts to essentially "go full steam ahead, but have more eval oversight before release".
More likely, this is being done to comply with SB 1119 (Adam's Law) [1,2] and similar legislation. These create extra steps that a service has to go through before releasing a new model that minors can interact with and exposes them to prosecution if a minor is harmed or harms someone else due to their interaction with a chatbot. It's safer for Anthropic to just exclude minors -- and they're unlikely to be paying customers anyway.
Thank you, I'm glad someone brought up SB 1119. I hate it.
California already has BPC §§ 22601-22606 to protect people from chatbots encouraging self-harm and I don't see a legitimate reason why they had to ratchet up the restrictions.
> Waymo says it has 170 million miles and I found 2 fatal accidents it was involved in (fault doesn't matter as fault is not included in the other stat either)
I suppose you can claim fault doesn't matter. But if you read the narratives of both fatal incidents, they are so clearly not the Waymo's fault that I don't see how you can hold it against them.
In the first [1], the Waymo was stopped at a red light when the car behind it was rear-ended by an SUV traveling at high speed. Both that car and the SUV then struck the Waymo. The SUV hit a total of 6 cars and someone in one of the other cars died. According to the police, the SUV may also have been involved in "multiple hit-and-run accidents reported just minutes before" [2].
In the second [3], the Waymo was yielding to a pedestrian when a motorcyclist made contact with the left rear corner of the Waymo. The motorcyclist fell to the ground to the left side of the Waymo, then another car tried to pass the Waymo and struck the motorcyclist, killing them. That other driver fled the scene.
If you drive on urban streets, no matter how good of a driver you are, there's a chance you're going to be involved in some sort of incident that's completely out of your control -- that seems to be what both of these are.
I agree that these cases are ridiculous to blame on the waymo entirely, but to give logic to their reasoning here you're unsure if on average the 1.3 fatalities per 100 million miles (FPMM) may also be on average ridiculously not the drivers fault as well. Perhaps if you were to throw out all ridiculous cases you'd have an average of CDL's having a 0.3 FPMM vs Waymo's having 0 FPMM.
The nuance you describe at the end is the better interpretation: fatalities per mile driven is a useless metric since it's such a rare occurance and incredibly circumstantial that it cannot be used as a meaningful comparison metric for safety.
It also gets better depending on the medical services nearby and there competence. So having more competent medical services in cities, lowers murder statistics and car-fatalities. Thus the actual accident rate would only be visible in the countryside.
If we want to get closer to the "real" number, we would also need to account for mechanic failures of the car, wildlife, and suicide. We don't usually attribute people jumping in front of trains as the fault of the person driving the train. Fatalities per mile driven may very well be biased in favor of waymo that has nothing to do with safety of drivers, which in turn create uncertainty when trying to estimate the statistical result of making all cars Autonomous. For example, car owners can have a higher variance in maintenance than waymo, they can use the car in areas with more wildlife than city traffic, and they can commit suicide through other means than personal being behind the wheel (suicide statistics is itself quite complicated).
Um, in those two examples where it was ridiculously not waymo's fault it was still absolutely the fault of a human driver. I'm struggling to imagine a significant amount of cases where it's ridiculously not the fault of any human driver.
Thats why he specifically called out CDL drivers, or in another part of this thread they talked about rideshare drivers. Certain segments of humans likely approach the same rates as the driverless cars.
The commonly cited fatality statistics do not discuss fault, so, absent more precise information, a apples to apples comparison should not conditionalize on fault.
Why? Well let us try to imagine a world where Waymo has the statistically average fatal crash rate, yet is never at fault. In this world, every fatal crash occurs at the Waymos. You just have a bunch of crazy drivers roving the streets causing fatal crashes against (or involving) unsuspecting, not-at-fault victim Waymos.
If that is the world, then are these crazy drivers only crashing into Waymos? Probably not, these crazy drivers would almost certainly be crashing into a statistically average distribution of unsuspecting, not-at-fault victims.
So this imagined world is consistent with the overwhelming majority of human drivers engaging in 0 fatal accidents at fault with a small, crazy roving subset being at fault for all fatal accidents. So, in that world, a Waymo that is only ask good as the statically average driver would actually be infinitely worse than the overwhelming majority of human drivers. If you only replaced the 10th percentile driver (90% of drivers are better), you would actually be increasing the fatality rate. Only by displacing that crazy subset would you actually be decreasing the fatality rate.
Are we in that world? The Waymo numbers on fatal accident involvement with no fault actually kind of point in that direction (though the mileage numbers are still too low to make a strong conclusion). Seemingly massive crash and injury reduction due to massive reduction of at-fault crashes, but the fatality reduction does not seem proportional. Another possibility is that the Waymo driving distribution is biased toward being in situation that result in more no-fault fatal crashes.
But, that is all just conjecture and thought experiments. Unless we have better data, we unfortunately should be restricting ourselves to apples to apples comparisons which, in this case, do not conditionalize on fault.
Though, to nitpick on the original post, I think that the 2 fatal crashes in 170 million miles (~85 million miles per fatal crash which is similar to the USA human average of ~83 million miles per fatality), is probably not the correct analysis. The average fatal crash is probably 2 vehicles and 1 fatality. So, the 85 million miles per fatal crash versus 83 million miles per fatality is probably fine in the crash vs fatality direction. But I am pretty certain you would actually need to double the 170 million to account for the "statistically average mileage of the other cars".
Otherwise, if we have exactly two cars in the world each with 100 million miles and they engage in one fatal crash with one fatality, then we would conclude that car 1 has 1 fatality per 100 million miles and car 2 also has 1 fatality per 100 million miles. Naively averaging them, we could conclude that the overall fatality rate is 1 per 100 million miles. Therefore, over the 200 million miles the two of them drove, there were 2 fatalitys. Obviously incorrect, so we almost certainly need to weight them by their relative proportion when aggregating them.
It wasn't the coverup of the crash. It's the fact that they massively exaggerated their capabilities.
For example, the reason they started with night trips was that their car could not differentiate between kids and adults so they preferred to start operation when kids are less likely to be presetn.
> The motorcyclist fell to the ground to the left side of the Waymo, then another car tried to pass the Waymo and struck the motorcyclist, killing them. That other driver fled the scene.
That second accident looks like it could be one on those typical cases of SDC causing accidents by behaving unexpectedly (like braking for a pedestrian where human drivers would have denied right of way)
I havent read the doc and I am not arguing either way, just saying that you can cause an accident in other ways than by ploughing into someone.
They really can't stop tying themselves into knots to dunk on self driving cars. They don't even know it's the case, they just suspect it's the case. Even if it _were_ the case, the drivers behind them most likely don't have this information available, they probably can't distinguish between "SDV being cautions and giving right of way" and "SDV stopping an immediate impact", they can just see a car in front of them suddnely stopping. So they should stop.
If the pedestrian hasn't gotten to the driveway quite yet, and you're stopping to let them get there and cross out of an abundance of caution, then it may not have been legally required for you to yield. And the motorcyclist behind you could very reasonably have seen the pedestrian too (and reasoned that the pedestrian was still several seconds away from getting to the driveway) and not expected you to stop. In such a situation, such an abundance of caution could have easily "caused" an accident, in that not yielding and proceeding through would have been perfectly safe.
(I'm of course being steel-manning GP's argument here, it could have very easily been that the pedestrian was already crossing the driveway at that point, I didn't see the video. Point is, "more caution" doesn't always mean "less chance of an accident".)
> Point is, "more caution" doesn't always mean "less chance of an accident"
that's a fair point. For example, if you're on a highway and traffic is moving at 80mph however the speed limit is 55mph and so you're driving 55 you're doing way more harm than good. Becoming an obstacle to traffic makes an accident more likely regardless of whether you're right or wrong.
Having said that, I do not see the waymo at fault here at all. The other car driver turned a minor accident into a fatality without involving the waymo, pedestrian, or even the motorcyclist. They were just not paying attention and ran over the poor person.
This is exactly the type of nuance that the comparison fails on. Plus we have limited info here.
Yes ofc we must yield to pedestrians. And we also make nuanced decisions. "There's a motorcycle moving too fast behind me. There's a pedestrian in front of me."
If the pedestrian would be killed or struck by the vehicle, then yes you need to stop and/or steer away. But if it's about merely inconveniencing the pedestrian to prevent an accident, that's the ethical duty.
That's the nuance the rule "always yield to pedestrians" needs to consider.
Hi. This position is utterly idiotic, even if you were just trying to steel-man it. If a self-driving car can "cause" accidents by yielding to crosswalks then the only sensible course of action is to immediately ban human drivers in cities, starting this morning, zero exceptions, no tolerance.
> No modern LLM can "reason" - otherwise you could give it a task like "make me money", it would ask you all the questions it needs about information that it doesn't know about and needs to know to make you money, then it would set whatever it needs to set up to make you money.
If that's your bar for reasoning, then most people can't reason either.
If the model includes unique data from a person then that person can identify the data - the allegedly plagiarised material - and so re-identify it. There doesn't need to be a privacy breach to close that loop as it requires the person to identify the information is associated with them first.
Or it could simply mean that they're operating at the limit of the capacity they were able to purchase and do not have headroom to handle load spikes. From what I understand, that's the situation Anthropic is in. And since Anthropic is now leasing a large portion of xAI's datacenter capacity, it's plausible that an Anthropic load spike could cause issues for xAI as well.
Your comment makes it sound like they can just push a button and spin up more capacity -- but at this scale and in this GPU-constrained environment, that's not really how it works.
Well signing a contract is more about bearing responsibility, even if you granted LLMs "personhood" they cant' meaningfully bear responsibility. So unless OpenAI is ok with having their C-suite face every consequence for what their agents do, including jail time, fines etc, then it doesn't matter.
reply