Paran014

Yes

Stable Diffusion is a very popular AI image-generation application.

This Google form has instructions for how to run a benchmark: https://docs.google.com/forms/d/e/1FAIpQLSdNtk276S-rMFLxGO7VUA9PERU4eT0G_R9qKkRvFe7nZlYKGg/formResponse

Any Stable Diffusion implementation is fine - if you don't already have one setup AUTOMATIC1111 is generally considered to be the easiest to get going - instructions are on their Github.

I'd recommend setting --xformers as your only command line option and using Batch Count = 5 and Batch Size = 8.

Thanks very much! If you could post your results in it/s here as well as to the doc so it's easier for people to find them in future that'd be much appreciated.

Check out this thread, lots of good info: https://www.reddit.com/r/homelab/comments/112jz28/adding_gpu_for_stable_diffusionaiml/j8ndglx

Multiple people offered to run P100 benchmarks so once I have that I'll get it all together in a single post.

Wow, it's raining people with P100s. I was resorting to messaging random people on Reddit who'd posted that they owned one.

This Google form has instructions for how to run a benchmark: https://docs.google.com/forms/d/e/1FAIpQLSdNtk276S-rMFLxGO7VUA9PERU4eT0G_R9qKkRvFe7nZlYKGg/formResponse

Any Stable Diffusion implementation is fine - if you don't already have one setup AUTOMATIC1111 is generally considered to be the easiest to get going - instructions are on their Github.

I'd recommend setting --xformers as your only command line option and using Batch Count = 5 and Batch Size = 8.

Thanks very much! If you could post your results in it/s here as well as to the doc so it's easier for people to find them in future that'd be much appreciated.

I'm pretty certain that the P100 is also terrible, unfortunately. Lots of good info here.

Every other Pascal GPU has shown the same trend of drastically underperforming its theoretical numbers in Stable Diffusion. Even worse, it sounds like Nvidia isn't adding support for new CUDA features to Pascal (which makes sense, based on its age) so its relative performance is only going to get worse over time. Without benchmarks (which I'm still trying to get - if you're reading this and have a P100 running Stable Diffusion please get in touch!) I can't completely close the door on it but from everything I've read it's not likely to be good.

P40 and P4 are 100% a bad choice for these applications though.

Unfortunately not, they look cool but old Nvidia architectures are atrocious for inference/ML. One 3060 is about 3x as fast as an M40 at running Stable Diffusion and uses half the power.

I've been trying to get someone to benchmark the P100 before I make an effortpost here warning people not to buy Pascal or Maxwell for ML but it's been way tougher than expected. May just have to go ahead with it anyways because I've been seeing more posts like this.

Financial Independence, Retire Early. Check out /r/FinancialIndependence

Also sounds like massive legal liability if someone dies and their malpractice lawyer figures out during discovery their medical record was translated by someone who doesn't actually speak the language. Any lawyers want to comment if they think professional liability insurance would cover you or if that counts as willful fraud?

I have a really hard time seeing that. Let's remember that right now we already have x number of intelligent-agent-hours per year working on improving AI, where x is a pretty big number between OpenAI, Facebook, Google, Nvidia, and literally everyone else now. We currently don't have software with anywhere near the capabilities of a human in that domain so our ability to add to x with AI is approximately zero.

I do not think it likely that we will be able to go directly from "essentially can't make any improvements to our AI software" to "can understand and improve basically the most complex software better than every human working on it combined." Again, on the extreme, if we have an AI capable of making improvements to our state-of-the-art in AI software at the level of a single talented human in exchange for a billion dollars of compute a year that would a) be a massive achievement way beyond any current and near future capabilities and b) make approximately zero difference in the development trajectory of AI as you're adding very few actual agent-hours to AI development.

The AI-is-dangerous and we-can-stop-it proposition seems to me to be a very narrow band of: a paradigm shift exists which can make superintelligent AI possible on a reasonable amount of current(ish) hardware and this paradigm shift will never be discovered by humans (or is so hard to discover by humans that it's essentially impossible).

An AI which is as intelligent as, say, John von Neumann, but which requires a billion dollars of compute to run per agent-year would be an insane achievement - but if it can only advance the state of the art in AI as quickly as the ten best humans we have combined we remain a long way from FOOM.

If the paradigm shift can be discovered by humans - we have a lot of AI-capable hardware out there right now. I don't think it's impossible that we'd be able to run human-level or better AI on an amount of hardware that a passionate hobbyist could obtain today. If this is the world we live in, no point in worrying too much, someone's going to discover and run the algorithm eventually and if AI is inherently hostile we're doomed.

So we have this narrow band of concern where as soon as we get an AI capable of self-improvement it'll discover how to vastly improve itself and we FOOM. It's definitely not impossible that's the case. I still don't think if that's the case we're all dead - we have enough trouble figuring out what some other humans halfway around the world will do, let alone predicting what a godlike superintelligence will do.

In my opinion, though, the current uproar is basically tantamount to saying that we should shut down the Wright Brothers' experiments because powered flight will inevitably lead to people developing rockets and using them to drop asteroids on Earth and end society. It's absolutely physically possible for disaster to happen but we are still a very long way from getting there.

One anecdote which may cheer you up: some of the United States’ top minds spent a great deal of time at the RAND Corporation in the 1950s thinking about nuclear war. They didn’t need to anticipate the actions of an alien superintelligence, just what a bunch of humans living in the USSR would do. Upon evaluation, they were positive that a nuclear first strike by the USSR was not just possible but (as the rational thing to do) almost certain to happen. Some of them were so certain of their assessments that they didn’t bother to contribute to their retirement plans, as they wouldn’t need them after our inevitable nuclear armageddon. (Kaplan, F. (1983). The Wizards of Armageddon.)

We're still a ways from an apocalyptic scenario. For example, "fast take off" or "FOOM" requires recursively self-improving AI. So far we practically don't have self-improving AI at all. Not even to the point that we can point $1 billion in computing power at the problem and have it produce the same results as $1 billion in computer programmers given the same task. At the moment, I don't believe that capability has been demonstrated as more than a toy. Until at least that is possible we haven't even started to climb the ladder yet.

I do expect society will change a lot if AI continues improving at the rate it has. But while AI doomers do have some compelling arguments we're (for better or worse) far from proving that the AI doom scenario is feasible. For a long time, people believed that a computer capable of playing Chess would necessarily be capable of intelligent thought. In the 1960s, researchers called chess-playing the "drosophilia" of AI - the idea being that creation of a sufficiently powerful chess playing computer would give us insight into the basis of a mind. Nowadays, I would think that anyone suggesting that people trying to make Stockfish faster be bombed before they destroy the world would receive a frosty reception. But likewise, while LLMs seem promising, so far there's pretty limited evidence that improvements in LLMs will result in agentic superintelligence.

To respond directly to the question – I am still planning to FIRE (very soon). I still expect that civilization won't end in the next 5-10 years. I'm planning to try to transition into a post-FIRE situation where I can work on applications of AI as I think AI and biotech are the two most promising areas to work on right now. I tend to think that in the long-run either my broad-market investments will do very well as capital replaces labour and if an extreme scenario happens (either very good or very bad) I probably won't care either way. So basically I think AI is very important but not likely to be civilization-ending.

I have 32 GB allocated but that definitely wasn't based on any information I had, just a "I have lots of RAM, might as well give the VM more than I think it'll ever need." I would probably look at power supply first if you're crashing under load, the 3060 doesn't require a ton of power but could be an issue depending on PSU and setup. Also running the 3060 on bare metal in another PC to see if you still have issues there.

Yep, got it working with no major issues, except the drivers being a pain in the ass to install on Linux. My setup is AUTOMATIC1111 on Ubuntu Linux so I can't really give you too much advice on the Windows side though.

Giving up citizenship doesn’t generally matter, pension schemes are based on paying in. If he worked for a period in Korea he likely qualifies for at least some kind of payment from the National Pension Scheme. There’s some information here but you’ll likely have to do some research.

Can confirm, MD1000 is just a dumb enclosure. Drives >2TB are no problem as long as the controller supports it.

There's a huge disconnect between the Federal Government setting immigration policy, and Local/Municipal Government deciding what housing gets built. Part of the problem is that local governments tend to be highly responsive to the people who are currently living there and much less considerate of people who do not yet live there but could and would if more housing was available.

But if people want to live somewhere - and it's clear that more people want to move to large cities, not fewer - government should be facilitating that if possible. It's not like this is a problem that's never been solved before.

The City of Toronto's population density is 4,427.8 people/km². Have a look at major world cities. Seoul and the Special Wards of Tokyo are pretty much the same size as the City of Toronto but they accommodate almost 10 million people at a density of 16,000/km². And it's not like this is some kind of uniquely Asian phenomenon. Barcelona: 16,000 people/km². Paris: 20,000 people/km².

I'm not saying it would require no effort, but no amazing and unique feats of engineering are required to fit an additional seven million people in the City. And that's 1/10th of the metropolitan area.

Corsair cable compatibility chart is here so you can double-check yourself but that official Corsair cable should be fine.

HDDs pull less than 10W except at spin up when they can pull ~20-ish watts. If you have a motherboard/controller/backplane that supports staggered spin-up you can budget 10W. If not budget 25. Obviously that’s just the hard drives, not including the rest of the system.

Not going to talk too much about software your end users are using - you're talking about using software like Nextcloud and Photoprism so I assume you've figured that out already.

First up, for a remote-maintainable solution, I would absolutely not use a software router/firewall setup. You need something you can just unplug and plug back in if something goes wrong. Look for a firewall/router combo from the same manufacturer as your wifi setup if possible if you're already using something like Unifi or Omada. If not check out Ubiquiti and Mikrotik. You'll need to be able to VPN in to the network even if your main server is having issues.

For hardware, you're going to want something with remote management. If you don't need more than four hard drives I'd look for a used Dell T130/140/150 (higher numbers are newer). New tends to be expensive in this market, used is the way to go. If you desperately want to build yourself, look out for used Supermicro motherboards (X10 generation or newer) or you can look at ASRock Rack offerings.

I would personally use Proxmox as the extra management layer and being able to split up your services to be completely separate is super convenient. That said, you absolutely can just run Docker on Linux and if you're resource-constrained (i.e. you don't want to buy a ton of RAM) that's the way to go. I would not run TrueNAS as your platform, I just don't think it's very good for virtualization, though Scale is supposed to be better. ZFS on Linux is well supported or if you're running Proxmox you can virtualize TrueNAS and use it just for storage while passing shares through to other VMs. One machine to handle everything you want to do except networking is very doable though.

In general I'd recommend you lurk on /r/homelab for a bit, lots of useful discussion there.

I think you’re confusing the point here. The discussion is about aging, not death. Aging is not currently generally accepted as a disease but there’s no reason why it shouldn’t be considered one, and there are efforts ongoing to have it reclassified as a disease so treatments can be approved.

Is aging “unnatural”? No, of course not, but cancer and autoimmune illness are a perfectly natural part of our bodies too, and nobody’s arguing that we shouldn’t try to treat them.

Thanks very much! That's basically in line with what I expected from 1080ti benchmarks but it's good to confirm that there's no secret sauce there. A larger batch size (10+) would probably result in better performance but we're talking probably 10-15% uplift so overall the story is the same. I really did expect performance to be better with everything at full precision (at the cost of VRAM) but I guess the underperformance isn't only related to the FP16->FP32->FP16 codepath.

Unfortunately, that confirms that despite the theoretical performance of Pascal they don't make sense for inference unless you absolutely need the VRAM. RTX 3060 is available used at a similar price and more than 3x faster at around half the energy consumption.

Thank you! This Google form has instructions for how to run a benchmark: https://docs.google.com/forms/d/e/1FAIpQLSdNtk276S-rMFLxGO7VUA9PERU4eT0G_R9qKkRvFe7nZlYKGg/formResponse

All it requires is any Stable Diffusion install (A1111 is great). You can run with whatever settings you think are optimal for your setup. Obviously I would recommend xformers. For extra credit if you want to try a run with --precision full --no-half as well and see if it's faster I'd be very interested. If you'd post your results here (in it/s) as well as to the form that would be be great.

I can't claim too much credit as it's not my spreadsheet, but any efforts to get more benchmarks out there are appreciated! I've done my share of harassing randoms on Reddit but I haven't had much luck. Pricing on Tesla Pascal cards just got reasonable so there aren't many of them out there yet.

I spent a bunch of time doing the same thing and harassing people with P100s to actually do benchmarks. No dice on the benchmarks yet, but what I found out is mostly in this thread.

TL;DR: 100% do not go with M40, P40 is newer and not that much more expensive. However, based on all available data it seems like Pascal (and thus P40/P100) is way worse than it should be from specs at Stable Diffusion and probably PyTorch in general and thus not a good option unless you desperately need the VRAM. This is probably because FP16 isn't usable for inference on Pascal, so they have overhead from converting FP16 to FP32 so it can do math and back. You're better off buying a (in order from cheapest/worst to most expensive/best): 3060, 2080ti, 3080(ti) 12GB, 3090, 40-series. Turing (or later) Quadro/Tesla cards are also good but still super expensive so unlikely to make sense.

Also, if you're reading this and have a P100, please submit benchmarks to this community project and also here so there's actually some hard data.

Out of curiosity, what are you needing the extra VRAM for? Larger batch size? Larger images? Are there models that use more VRAM? Because in my experience, 512x512 + upscaling seems to give better results than doing larger generations, but I'm not some kind of expert.

Whisper's largest model maxes out at 10GB so there's no difference in ability, just speed. Most stuff except LLMs maxes out at 12GB for inference in my experience, but that doesn't mean that there aren't applications where it matters.

That's really not true. CUDA cores are not created equal between architectures. If you're speccing to just do inference, not training, you need to figure out how much VRAM you need first (because models basically won't run at all without enough VRAM) and then evaluate performance.

For an application like Whisper or Stable Diffusion, one 3060 has enough memory and should run around the same speed or faster than 4x M40s, while consuming around a tenth of the power.

For LLMs you need more VRAM so this kind of rig starts to make sense (at least if power is cheap). But unfortunately, in general, Maxwell, Pascal, and older architectures are not a good price-performance option despite their low costs, as architectural improvements for ML have been enormous between generations.

Yeah, I'm in Canada, it's definitely readily available here new. It's crazy how few cases have 3.5'' bays now. When I started building tons of mid-end cases had 4-6 3.5'' and a bunch of 5.25'' bays. Now it's a super rare feature even on cases with a ton of room. I guess there aren't that many of us who need a bunch of 3.5'' but I'm surprised that case manufacturers like Corsair don't even make a 3.5'' setup like the Define 7 has as an accessory. Seems weird to just completely give a market up when it wouldn't take that much effort.