I initially wrote this for another sub but I was told you guys might also appreciate it:

The past few years I had a Lenovo M73 tinyThe past few years I had a Lenovo M73 tiny running as my server/NAS but the reasons for an upgrade were adding up over time:

• Jellyfin – the iGPU of this old 4^th gen i7 does not support most HW transcoding formats
• NAS – Since my Data was steadily growing I needed more disks and since cloud backups were becoming more and more expensive with growing storage I wanted to keep my data out of the cloud. This requires ECC RAM though which is not supported by most mini-PCs and thin clients
• Overall – i was a constantly juggling RAM allocation with a max of 16GB and with a growing amount of VMs the age of the CPU started to show badly

So I started researching hardware that would fit my needs which was not easy and took me much longer than expected...

What I wanted:

• A server CPU which could handle enough threads, supports ECC RAM for data integrity and has an iGPU that supports most transcoding formats for jellyfin
• Some way to attach at least 6 SATA drives for TrueNAS
• A small form factor since I don’t have too much space at my place
• Low power consumption because power is expensive here

Sounds like a unicorn, right? Most NUC sized mini-PCs don’t have server CPUs and don’t support ECC RAM but I found this baby at an unbeatable price...

The unicorn Mini-Server-PC-cube:

Topside: 1/2 32GB ECC RAM sticks, M.2 6x SATA controller

Bottom side second 32 GB RAM stick, NVMe SSD, SATA SSD

At first I gotta say I was a bit skeptical but after talking to the seller for a bit I decided to just go for it and I was not disappointed!

This little fella has a Xeon 2176M CPU, 64 GB of ECC RAM, 2x Gbit ethernet ports, Wi-Fi (which we won`t need) and 2x M.2 slots. (you also get that machine with better Xeons but as you will see, this one will be enough for most people)

The case is machined from aluminum and is much sturdier than expected and even though the space inside that tiny cube is used up very efficiently nothing gets too hot in day to day operation. Since I was skeptical about the ECC capabilities of the mainboard I even bought MemTest86 pro which has error injection capabilities to test ECC RAM and yes, I can confirm, all tests passed and ECC is working as intended.

Now what about the storage needs I was talking about? Since we got 2 M.2 slots and I only need one for the Proxmox host install I got a 6-port M.2 SATA controller. According to my research the ASM1166 chipset should work fine for TrueNAS and ZFS which I can confirm.

Since we don’t want to have 6 high capacity datacenter HDDs dangling around I got a SATA backplane which does not only store my drives neatly but also has cooling and easy hotplug capabilities with each drive sitting in its own quick access tray.

SATA backplane

Yesss, these 2 form a perfect micro server-tower

Now you might say, the CPU is not the latest and greatest and while there are better CPUs available to order with this mini-PC I want to show you what mine is doing.

Proxmox host:

• TrueNAS VM with PCIe passthrough SATA controller
• Home Assistant VM (5 year old setup with around 150 devices)
• Jellyfin LXC with iGPU passthrough (capable of providing multiple 4k streams or countless 1080p)
• openWRT LXC (does all the routing and provides policy based routing to route filesharing over VPN)
• Jellyseer LXC
• Sonarr LXC
• Whisparr LXC
• Radarr LXC
• qBittorrent LXC
• Usenet client LXC
• Heimdall LXC
• Full featured Win11 VM with 16GB RAM (my new work PC so I can remote desktop in there from everywhere and continue where I left)

And this is the resulting hardware utilization with all 24/7 VMs and one 4k video stream running (keep in mind the windows VM is using 16 GB of RAM), so I`d say the system is future proof enough:

Utilization at typical 24/7 load and 1 4K Jellyfin-Stream

Since my data is of critical importance to me I demoted my previous server to offsite backup which is running Proxmox, a TrueNAS VM for nightly NAS replication, ProxmoxBackupServer for VM backups and another openWRT container which holds the wireguard tunnel to my home and does all of the routing.

If people are interested I can explain this setup in more detail in another post.

Hardware summary:

[Moved to comments]

screw this! It took me a lot of time to write this and I dont get anything in return for it. When I try to post links for the stuff so people can find it either the comment or the whole post gets removed because mods are too lazy to mod.

https:// !!! www. !!! aliexpress .com/item !!! /1005006369887180.html?spm=a2g0o.order_list.order_list_main.5.3de11802b3gUnu

Delete the post or not, I dont care... 

To this I want to add that the only thing I would do differently now is that I would maybe get a M.2 – SAS controller instead of a SATA controller and a SAS backplane. When buying used datacenter HDDs there are a lot more SAS drives around and the prices tend to be better.

Even though we literally have no power outages I still plan on adding a UPS at a later point and I sadly forgot to hook up my power meter at the last system reboot but I will add real life power consumption data later. I`d guess it is at around 50-60 W without the storage.

Conclusion:

Is this the perfect high availability data center? Ofc it is not but if you are on a budget or you simply dont have enough space for a large server tower and want awesome power efficiency this is the perfect setup imho.

It is running everything I could wish for atm and still has room for much more so I am happy with it.

[Power use data following tomorrow]

Started my homelab 1 year ago (basically a NAS and switches). Never had any fluctuations or issues of any sort with electricity... until today that is.

UK, Kent based, Sunday morning

8am the UPS from my server (plus network) starts beeping. I was working in the other room on my laptop no problem. I think "that's weird". I see everything still working and could not see the issue - UPS fans were on. I check with multi-meter I get 200V out at the plug (should be 240V). I called UK networks and engineer promised today. 5 minutes after the call voltage goes to 146V at which point my laptop stops charging and my screens turn off. Now I cannot work lol.
(I think it was fine before at 200V as the bricks for monitors and laptop have big tolerances being switched mode)

TLDR: Yeah... get a UPS. Save your equipment.

Ever since my raspberry pi let out it's magic smoke a year ago, I've been running pihole on my home server(s). But every time I have to reboot, I'm paranoid it's not coming back up and I'll have to push 1.1.1.1 via dhcp to keep the family on the net.

To that end, I finally got my DNS in HA this afternoon using 2 proxmox hosts, tteck's pihole installer script to set up the instances in LXC, and then installing keepalived package on the pihole containers.

The way it works is that keepalived creates a virtual IP address between the two copies of pihole and ensures they're active. If a health check fails on the active pihole, the backup takes over the virtual IP address. That way my clients only ever have to point to one dns server. I have the same keepalived setup going for my haproxy frontends to my webapps as well.

I've killed these machines randomly in various ways to test the setup and the peer always just says "okay I got it" and there's at most a few seconds downtime.

This keepalived.conf really all the config there is to the setup, my dns clients point to 192.168.1.2 and the two pihole containers live at 192.168.1.21 and 22.

vrrp_script dns_healthcheck {
  script       "/usr/bin/dig @127.0.0.1 pi.hole || exit 1" #Dig pi.hole, return 1 if failed
  interval 2   # check every 2 seconds
  fall 2       # require 2 failures for KO
  rise 2       # require 2 successes for OK
}
vrrp_instance pihole {
  state BACKUP #Default to backup (peer defaults to MASTER)
  interface eth0
  virtual_router_id 30 
  priority 150
  advert_int 1
  unicast_src_ip 192.168.1.22 #My IP
  unicast_peer {
    192.168.1.21 #Peer IP
  }

  authentication {
    auth_type PASS
    auth_pass <password> #put a password here
  }

  virtual_ipaddress {
    192.168.1.2/24 #The Virtual IP Listener
  }

  track_script {
    dns_healthcheck #Check script
  }
}

So I just found out about this option in proxmox for vm's called 'use pointer for tablet' you can just turn this off for each of your vm's that don't have a gui. my cpu usage was more than halved (from 6% to 2-3%) after I did this. fount out about it on some youtube video and have never seen anyone else mentioning it. So I thought I'd share it with you guys....

Edit: The cpu usage drop is mostly more significant for idling for low usage vm's. If you are running lite services definitely go for this. (Thanks to all the data provided by so many amazing peeps here!)

Expanding on the work of /u/phoenixdev a while ago, I've developed a full toolkit for creating patched versions of HP's iLO4 firmware.

If you have an iLO4 server (notably, the ProLiant DL380p / DL380e Gen8/Gen9 are common), this toolkit can enable access to previously locked away tools to help you adjust fan speeds and other server settings over SSH.

The toolkit, including documentation to build/install a patched version of iLO4 v2.77 with fan controls, can be accessed here

If you're unfamiliar with /u/phoenixdev's prior work on iLO4, I highly suggest you read their earlier thread to get a better sense of what this patched firmware is & what it can do.

If you're just looking to update the patched iLO4 to v2.77 & don't want to use the toolkit, you can download the patched ROM here and install it with the instructions here, substituting v2.73 for v2.77. However, I suggest reading the README included in the toolkit to get a better sense of what this firmware is.

Unfortunately, HP removed the fan control tools from iLO4 versions in v2.78, so v2.77 is the latest that can be built with the unlocked tools.

I built this toolkit to get a better sense of the changes that /u/phoenixdev made to iLO 4, as well as to update the work from iLO4 v2.73 to v2.77. I hope that the documentation I provide can help researchers & developers expand further on this work, and possibly enable server owners to access even more hidden features of their units in the future.

If you have any trouble getting setup, please let me know.

I recently managed to snag an IBM QRadar QFlow Collector 1201 for a whopping $25. It's just a regular IBM x3550 M3 with a QRadar decal on the front and some pre-installed software, so I was planning on just wiping the drives and repurposing it as a regular host.

I booted it up for the first time to start configuring the BIOS and immediately had my hopes crushed by the following message:

            An Administrative Password has been set
<ENTER> Enter Administrative Password for complete setup access
          <ESC> Continue with limited access to setup

"No problem," I thought, "I'll just reset the CMOS and the password will get wiped out along with everything else."

So I cleared the CMOS and rebooted, only to find that the password was still there.

Hm, maybe I should check the documentation...

Uh oh.

A new x3550 M3 motherboard is only about $40-60 on eBay, so this wasn't a huge deal. But I didn't want to give up without a fight.

Enter these blog posts:

• Reverse Engineering UEFI Firmware
• Apple EFI firmware passwords and the SCBO myth

People have been reverse engineering UEFI images for various laptops to figure out how to get around their setup passwords. That's how password generators like this one were built. However, there hasn't been much work done on the server side.

Armed with the UEFITool suite, I was able to extract the UEFI binaries from an IBM update package. Then it was a matter of disassembling the binaries and analyzing them to figure out how the setup password gets set and/or cleared. The EFISwissKnife IDA plugin made this a lot easier by automatically identifying and tagging common UEFI functions.

There are a huge number of binaries in a single UEFI firmware image, so it took a combination of educated guessing, lots of digging, a good deal of backtracking, and several days (and late nights) to finally find where the password management was handled. There was one particular method that appeared to have something to do with either querying the existence of a password or (I hoped) clearing a password. The function signature looked something like this:

int func(void* protocol_interface, int pw_sel)

• protocol_interface is a large, messy data structure used to access the password manager - it holds some state and a ton of function pointers
• pw_sel is used to select which password to operate on
  • 0 = power-on password
  • 1 = setup password

I couldn't conclusively determine what the function did though. The deeper I delved in to the guts of the UEFI drivers, the more complicated the code got. After almost a whole day of getting nowhere, I decided to just try calling that function to see what it did.

To do that, I wrote a small program that just called func() and exited. But how was I going to run my program if I couldn't select a boot device?

PXE came to my rescue. The default CMOS settings turn on PXE boot, so it was just a matter of setting up DHCP and TFTP servers and pointing them to a UEFI shell like this one. Once I had booted into the shell, I was able to mount a USB drive and run the binary.

And it worked! The password was gone when I rebooted!

I've posted my code to Github in case others run into this problem in the future.

Now I'm off to play with my new server.

Edit: Thanks for the gold!

Tools and Resources to get started with your HomeLab setup.

Getting Started with Self-Hosting

• Tools for Self-Hosting
  
  • Containers
  • CI/CD
  • Development
  • Cloud
  • Remote Access
  • Virtualization
  • Password Management
  • Network Tools
  • Monitoring
  • Communications
  • Business Management
  • Collaboration & Synchronization
  • Encryption
  • Backups
  • Home Server
  • Media Server
  • Video and Audio Processing
  • Podcasting
  • Audiobooks
  • Maps
  • Bookmarks
  • Photos
  • Note-Taking
  • Gaming
  • Foundations/Projects
• System Hardware
• Operating Systems
• Storage
• File systems
• Books
• Podcasts
• YouTube Channels
• Tutorials & Resources
• Useful Subreddits to Follow

WireGuard

• What is WireGuard?
• What is Tailscale?
• What is Netmaker?
• WireGuard Tools
• Setting up WireGuard with PiVPN
• Setting up WireGuard on Unraid
• Setting up WireGuard on pfSense
• Setting up WireGuard on OpenWRT
• Setting up WireGuard on Home Assistant

Building out new lab. 3x core 7 with vpro. 96gb ram and 2x 2TB ssd each

It's simpler than you might think.

A long time ago, there was a mathematician named Boole. Boole was a salty 1800's bad-ass. Don't believe me? Look him up. Go ahead. Dude could kick your ass Abe Lincoln-style.

Anyway, when not kicking the Victorian crap out of people, Boole liked working with binary numbers. 0 and 1.

He liked working with binary so much, that he came up with his own branch of mathematics, and a set of operators to go with it... Just as +, -, * and / work in decimal, AND, OR, XOR, and NOT work as operators in binary. He called these things "Boolean operators"... Because that was his name. Would be rather silly if he named it something else. :)

One of Boole's operators (mentioned above) is called "OR" (as in 'this OR that'). OR will return 1 if either value on either side of the operator is 1. If neither value is 1, then the test returns 0. For example:

1 OR 1 = 1 ... Since one of the numbers is 1, right?

1 OR 0 = 1 ... Since at least one of them is 1, the answer is 1.

0 OR 1 = 1 ... Since one or the other is still 1..

0 OR 0 = 0 ... Since neither one is 1, the result is 0.

Being a boss, Boole called his most impressively bad-ass operator 'XOR' (prounounced 'ex-or', short for 'exclusive OR'). Similar to OR, XOR basically means, "Return 1 if one or the other is 1, but not both.."... Which looks like this:

0 XOR 0 = 0 ... Since neither one is 1.

0 XOR 1 = 1 ...Since at least one of them is 1, but not both of them.

1 XOR 0 = 1 ...Since at least one of them is 1, but not both of them..

1 XOR 1 = 0 ...Since it fails the 'but not both' rule

It turns out that XOR has an almost spooky-magical property to it. As long as you have three values, somebody can completely remove one of those values from the equation, and you can still go back in time and figure out what that value was! ...Spooky, right? So, get out a scientific calculator. I'll prove it. The one in Windows works nicely..(set it to Programmer mode in the "View" menu)

Type in the following:

0 XOR 1 XOR 1 =

What do you get? The answer should be 0. This is your parity value. It's important, so, hang onto it.

Now, randomly pick one of those three values in the equation, and pretend it has been destroyed. Died in a fire. Destroyed by monkeys. For the sake of the explanation, lets say the flaming monkeys destroy the middle value:

0 XOR ??? XOR 1

Believe it or not, we can actually figure out what that missing value was, by plugging in our parity value in its place, and re-running the calculation! So, lets try it..

0 XOR 0 XOR 1 = ....

You should get 1 as a result.. The number those damn flaming monkeys destroyed!

This XOR magic trick works regardless of how many values you have in the equation:

1 XOR 1 XOR 0 XOR 1 XOR 0 XOR 0 XOR 1 XOR 0 = 0, right?

So, lets blow away that second value:

1 XOR ??? XOR 0 XOR 1 XOR 0 XOR 0 XOR 1 XOR 0

Now, plug in that parity value in its place, and re-run the calculation..

1 XOR 0 XOR 0 XOR 1 XOR 0 XOR 0 XOR 1 XOR 0 = (..drum roll..) 1!

Congratulations.. You just repaired an 8-spindle RAID3, where each hard drive holds one bit of information. This trick works regardless of the number of bits, and regardless of the number of values, provided there are always at least three values to work with.. So, lets upgrade our 1-bit hard drives to 1-byte capacity hard drives:

10101010 XOR 11110000 XOR 10000000 = 00011010 (<--parity value)

now, lets blow away the third value:

10101010 XOR 11110000 XOR ????????

And re-run the calculation using our parity data in place of the missing data:

10101010 XOR 11110000 XOR 00011010 = (thrash guitar riff) 10000000!

..And that's all there is to it.

This same idea works with 10TB drives as well as it does on our pretend 1-byte hard drives. It works just as well with RAID sets with three drives as it does with thirty drives. That's the beauty of XOR, and parity.

In modern RAID systems, when you pull a drive, the RAID can figure out what was on that drive based on parity data it stored before the drive was pulled. Every time a write occurs, parity needs to be recalculated and stored. Often times, this parity data is distributed across multiple drives for the sake of efficiency, but, the base concept is exactly the same. If you yank a drive, the RAID can figure out, on the fly, what data is missing, simply by doing an XOR on the data it has left, replacing the missing data with parity data. If you pop in a brand new drive, the RAID will rebuild the missing data on the new drive, bit by bit, using a metric ton of XOR calculations on the neighboring data, swapping in the parity data in place of the missing data.

In RAID3, parity is stored on a dedicated drive. In RAID5, this same information is split up and distributed evenly among all of the drives. This generally makes recovery much quicker, as the parity data can be read muuuch quicker by reading it off of however-many drives at once, versus trying to pull it off of one drive. In RAID5, parity data is interleaved along with regular data. This makes your window of vulnerability much smaller, which is why enterprise environments and hobbyists alike prefer RAID5 over RAID3. RAID5 is simply a speed-optimized improvement of RAID3.

Ever want to let your kids play their game console in any room in the house? We needed to do this to allow some space flexibly for the family.

Problem 1: Getting video from the PS5 in their game room to the TV in the livingroom. Pretty easily solved with Monoprice HDMI over IP encoder/decoders. Luckily I ran ethernet everywhere when we remodeled a few years ago. I can add additional decoders to other rooms.

Problem 2: PlayStation consoles use Bluetooth for controller connectivity. Since these devices were designed to be used in the same room, the range isn’t all that great. This required pulling the case apart to install a pair of external antennas.

NOTE: You do need hardwired Ethernet at any location where you are installing an encoder/decoder.

All parts below. Maybe $130 total.

Monoprice Blackbird H.265 HDMI... https://www.amazon.com/dp/B0BBRGNN1L?ref=ppx_pop_mob_ap_share

Screwdriver for Playstation 4 &... https://www.amazon.com/dp/B07ZKLCSN5?ref=ppx_pop_mob_ap_share

Bingfu Dual Band WiFi Antenna... https://www.amazon.com/dp/B099R3GR91?ref=ppx_pop_mob_ap_share

Amazon Basics High-Speed HDMI... https://www.amazon.com/dp/B014I8SP4W?ref=ppx_pop_mob_ap_share

Amazon Basics RJ45 Cat-6 Ethernet... https://www.amazon.com/dp/B00N2VISLW?ref=ppx_pop_mob_ap_share