Filtrar por género
Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as a platform for support and questions. We love and advocate FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and TrueOS. Our show aims to be helpful and informative for new users that want to learn about them, but still be entertaining for the people who are already pros. The show airs on Wednesdays at 2:00PM (US Eastern time) and the edited version is usually up the following day.
- 919 - 681: UNIX still powering the worlds infrastructure
Where Unix runs today, Webzfs updates, A remote filesystem for 2.11 BSD, Sylve, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Where Unix runs today
(Dont get distracted by the game)WebZFS Updates
The bugs are coming from inside the house NetBSD 11 SupportNews Roundup
A Basic Remote Filesystem for 2.11BSD
Curly braces: An evolution of UNIX and C
Sylve: FreeBSD bhyve Virtualization with Ansible Automation
Beastie Bits
1978 Aussi Users Group Newsletter Terminal UI and CLI for FreeBSD NFSv4 ACLs Sounds of the IBM 1401Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
[Jakob - Is this question too political for the show?]
[Reece - Webzfs]Hoping you lads can help solve a debate at work in the IT department. Whats the best song about a former Australian colony? Toto - Africa Men at work - Down Under For completeness: Split Enz - Six Months In A Leaky Boat (New Zealand) Don McLean - American Pie U2 - Sunday Bloody Sunday (Ireland) Gorillaz - Hong Kong Stan Rogers - Northwest Passage (Canada)Guys, Just following directions. Im interested in hearing more about it? Did you just say that because jt is a little shy if I remember correctly? Thanks for another great show and taking the time to do it. 73..Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 17 Sep 2026 - 1h 00min - 918 - 680: What FS?
BSD Part Deux, Code that built the internet, and an interview about WebZFS
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
BSD pt Deux
Interview JT on WebZFS
TJ: I started by asking JT...
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 10 Sep 2026 - 44min - 917 - 679: It also goes to 11
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
SR-IOV Is a first class feature
News Roundup
Grafana dashboards are a marvel of the modern web environment
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 03 Sep 2026 - 1h 02min - 916 - 678: Ceiling Cat has another meaning
Switches in your ceiling, AI affecting interest in things, Commodore Phone and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
How you wind up with switches above your office's false ceiling
News Roundup
Attempting to reproduce the missing etcmerge from 15.0 to 15.1 upgrade
Commodore is releasing a flip phone running Sailfish OS!
Why a Flip Phone? $399 - $640 preorderHardenedBSD June / July 2026 Status Report
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 27 Aug 2026 - 47min - 915 - 677: Butler at your service
Home Assistant Setup, OpenBSD Updates, Wine 11.14, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Huge!!! OpenBSD Updates
WPA3 support coming to OpenBSD Game of Trees 0.127 Released OpenBSD relayd(8) adds ECDSA support with CA engine code from smtpd(8) httpd(8) gains support for custom HTTP headers LLVM toolchain coming to OpenBSD/sparc64 Call for testing: OpenBSD vmm(4)/vmd(8) fd-ificationNews Roundup
Wine 11.14 Brings New WoW64 Mode to FreeBSD
I Built a FreeBSD Cloud to Use with FreeBSD
How Unix Spell Ran in 64kB RAM
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Tyler - IPv6 Question Reese - WebZFSSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 20 Aug 2026 - 1h 10min - 914 - 676: 10 print "goto 10"
The Future of the FreeBSD Kernel LLDB Plugin, Stop Ruining "10 PRINT", BoxyBSD Returns to FreeBSD, The Apple Lisa inside an FPGA, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Future of the FreeBSD Kernel LLDB Plugin
Please, Just Stop Ruining "10 PRINT". Seriously.
News Roundup
Google Summer of Code 2026 Reports: Testing Compat Linux: Syscall testing
The Apple Lisa inside an FPGA!
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Marcus - Linuxulator vs Bhyve- Reese - Feedback on episode 673
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 13 Aug 2026 - 1h 00min - 913 - 675: Going GPL Free
FreeBSD 16 goes GPL Free, The computer at the bottom of the lake, FreeBSD's new Board member, Phaethon, a new 68010 based Unix machine and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The Computer at the bottom of a canal
News Roundup
FreeBSD Foundation Welcomes New Board Member: Dave Cottlehuber
Phaethon 1 - a 68010-based Unix machine
Add a new m68k port oriented towards home-brew m68k machines
Bringing Swift to the Apple II
Working around dragons with the Lemote Yeeloong laptop and OpenBSD
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Producer note : Ive been insanely busy this past week and I've fallen behind on a few things, including checking the show email. If you sent an email in and we havent covered it yet, we'll get caught back up soon. If yuo haven't sent in an email... shame on you. You should email us and ask us something.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 06 Aug 2026 - 39min - 912 - 674: Software development never changes
Scrub Design, WebZFS Updates, The Foundations role in the FreeBSD Ecosystem, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
If Scrubs Hurt, Your ZFS Design Is Broken
WebZFS Updates
Beta Announcement Why wont my pool export zfs iostat- Pictures so you can get a better idea of how it works
News Roundup
Understanding the Foundation Board’s Role in the FreeBSD Ecosystem
Monitor your devices with LibreNMS on FreeBSD
Diskless Workstations Two Models
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 30 Jul 2026 - 1h 00min - 911 - 673: There's somebody at the door
Leaving Port 22 open to the internet, FreeBSD Foundationals, GhostBSD Finance Report, Making your own read-only device with NetBSD, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
I Left Port 22 Open on the Internet for 54 Days. Here's Who Showed Up.
FreeBSD Foundationals: The Boot Process - From the Loader to Boot Environments
News Roundup
GhostBSD - March 2026 Finance Report
Make your own Read-Only Device with NetBSD
NFS-problems I didn’t expect at all
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 23 Jul 2026 - 50min - 910 - 672: Kitchen Calculator
Unix Pipes under Load, Powering up a IBM Calculator from 1948, FreeBSD AI assisted Vulnerability Discovery Project, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Unix Pipes Under Load: Streaming, Barriers, Backpressure, and Bottlenecks
Powering up a module from the IBM 604: an electronic calculator from 1948
News Roundup
FreeBSD AI-assisted Vulnerability Discovery Project launch
Sometimes it actually is the network: a war story
FreeBSD Tribal Knowledge: Boot Enviroment Management
Beastie Bits
OpenSSH 10.4/10.4p1 released! OpenBSD/amd64 kernel virtual address space is now 512GB OpenBSD's pledge(2) and unveil(2) are developer-friendly, study findsTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 16 Jul 2026 - 47min - 909 - 671: Rage Against the Machine
The 40 Most Rage-Inducing Problems in Tech, ZFS vs Cep, Detecting and removing dangerous secrets on dev workstations before Shai-Hulud does and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The 40 Most Rage-Inducing Problems in Tech
ZFS vs Ceph: Do You Actually Need Ceph?
News Roundup
Detecting and removing dangerous secrets on dev workstations before Shai-Hulud does
syslogd(8) privileged and non-privileged parts now separate binaries
Getting Victoria Logs running on FreeBSD
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 09 Jul 2026 - 53min - 908 - 670: Failure is not an Option
How NASA built Artemis II's fault tolerent Computer, Jails vs LXC, Respsectfully getting a personal copy of a website, Opensense 26.1.9 and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
How NASA Built Artemis II’s Fault-Tolerant Computer
Jails vs LXC: What’s the Right Choice for Infrastructure?
News Roundup
FreeBSD 15.1-RELEASE Announcement
Release NotesNotes on respectfully getting a personal copy of a website's contents
Corrupting a ZFS File on Purpose
The status of OpenSSL 4.0 support in syslog-ng
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 02 Jul 2026 - 58min - 907 - 669: Poudriere Speed Run
inotify in FreeBSD, how changes to poudriere.conf affect the build time, Migrating mail servers from exim to OpenSMTPD, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
News Roundup
How changes to poudriere.conf affect the build time
Follow on Giving poudriere a jump startMigrating mail servers from exim to OpenSMTPD (smtpd) is fun and useful
Recap of the April 2026 Frankfurt Area FreeBSD Hackathon – Sven Ruediger
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 25 Jun 2026 - 37min - 906 - 668: Wiring up the BSDs
FreeBSD to OpenBSD Wireguard, Object storage with OpenZFS and SeaweedFS, a zfs script for labeling drives, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Using Object Storage with OpenZFS and SeaweedFS
News Roundup
zfs – a helper script for labelling all those drives
Creating a Samba Active Directory Domain Controller on FreeBSD
Beastie Bits
Let's find out how to get predictable IPv6 addresses assigned to OpenBSD VMsTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Davi - BSDCan 2026Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 18 Jun 2026 - 45min - 905 - 667: Don't exceed by security boundary
.NET on FreeBSD 15, Klara and TrueNAS fixing dedup, dhcpcd and unbound in FreeBSD Jails, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Running .NET 10.0 on FreeBSD 15.0
How Klara and TrueNAS collaborated to fix one of ZFS's longest standing limitations
News Roundup
dhcpd and unbound in FreeBSD jails
How our environment still needs the security boundary of Unix logins
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 11 Jun 2026 - 47min - 904 - 666: Everyone gets an LPE
fatgid, why zfs is ideal for media production, the CTF scene is dead, private repo behind TLS, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Why ZFS is the ideal filesystem for multi-user media production
News Roundup
A Private pkg Repo Behind Mutual TLS
This blog ran on Ubuntu 16.04 for 10 years. I migrated it to FreeBSD
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 04 Jun 2026 - 1h 04min - 903 - 665: 60 Puffies
OpenBSD 7.9, Critical Infrastructure in FreeBSD, GhostBSD Finance report, Solaris 11.4 updates, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
OpenBSD 7.9 60th Edition has been released and Reported over on Undeadly
Cleaning Up Critical Infrastructure in FreeBSD
News Roundup
Apple Wants to Kill Your Time Capsule but They Run NetBSD So They Can Not
Oracle To Reduce The Frequency Of Solaris 11.4 Updates
FreeBSD on a Thinkpad T14 Gen 2 Intel
Beastie Bits
The DragonFly site has a recently-updated page describing how DPorts is assembled and the process to contribute. TUHS - Unix use of VAX protection modes Origin of the rule that swap size should be 2x of the physical memory- The Duke and the Beastie - Improving OpenJDK support for FreeBSD
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 28 May 2026 - 1h 00min - 902 - 664: No one misses SPARC
The NetBSD/FreeBSD Merge announcement, the rise and fall of SPARC, GhoseBSD 26.2 and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
NetBSD/FreeBSD will not merge, November 1993 announcement
Rise and Fall of SPARC: Why No One Misses It
News Roundup
Help needed testing GhostBSD 26.2
Redundant DHCP server and DNS Resolver using OpenBSD and FreeBSD
Universities And In house Tech
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Paul - FeedbackSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 21 May 2026 - 1h 02min - 901 - 663: Proxhyve
Switching from Proxmox to Sylve, FreeBSD Quarterly report, FreeBSD's laptop program, Migrating ZFS, Haiku and OpenSSL news, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
I Switched from Proxmox to Its FreeBSD Counterpart on My Home Server – Here is How it Went
The FreeBSD Foundation's Laptop Support Project
News Roundup
Migrating ZFS filesystems from one zpool to another – same host
Haiku Isn’t Just For X86 Anymore, Boots On ARM In QEMU
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 14 May 2026 - 1h 01min - 900 - 662: I need a hero
Cybersecurity Looks Like Proof of Work Now, Compensating for RAM Constraints with L2ARC on ZFS, GhostBSD 26.1, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Cybersecurity Looks Like Proof of Work Now
Compensating for RAM Constraints with L2ARC on ZFS
News Roundup
I connected a phone to my FreeBSD server
Beastie Bits
BSD Can Schedule up OpenBSD Campaign 2025 OpenBSD Campaign 2026Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 07 May 2026 - 51min - 899 - 661: Break up Big Tech
Breaking up Big Tech, Porting MacOS to the Nintendo Wii, OpenBSD on the Pomera DM250, Postgres is your friend and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Porting MacOS to the Nintendo Wii
News Roundup
Installing OpenBSD on the Pomera DM250
Postgres is Your Friend. ORM is Not
I like to use Soviet control panels as a starting point
Beastie Bits
OSHintosh - an open source 68000 Macintosh Time to update 2.11BSD: biggest patch ever landed before 35th anniversary A quick and easy Guide to TmuxTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Producer Note, If you have emailed in and you havent heard back and we havent covered your message, email again. Our email is flooded with spam and I might have missed your message.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 30 Apr 2026 - 46min - 898 - 660: I just work here
Proxmox to FreeBSD, Hidden values of CPU-Intensive Compression, Cells for NetBSD, OpenBSD 7.8 on RPIs, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
From Proxmox to FreeBSD and Sylve in Our Office Lab
The Hidden Value of CPU-Intensive Compression on Modern Hardware
News Roundup
Cells for NetBSD – Kernel Enforced Jail Like Isolation with User Friendly Operations
OpenBSD 7.8 on Raspberry Pi Zero 2W
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Tim - Are OCI Images useful for Freebsd.md
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 23 Apr 2026 - 43min - 897 - 659: Full traffic send
Wayland setting back Linux, Dr Callahan's semi retirement, holding onto your hardware, PF queues breaking the 4gbps barrier, and mroe...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Wayland set the Linux Desktop back by 10 years
Semi-retirement, or, really, changing my relationship with the BSDs
[Hold on to Your Hardware](https://マリウス.com/hold-on-to-your-hardware/)
News Roundup
PF queues break the 4 Gbps barrier
Nobody said there was math on this exam!
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 16 Apr 2026 - 1h 08min - 896 - 658: It’s the vibe of it
FreeBSD and OpenZFS in the Quest for Technical Independence, Reviews make you 10x slower, OpenBSD on a Motorola 88000, Jailrun, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
FreeBSD and OpenZFS in the Quest for Technical Independence: A Storage Architect’s View
Every layer of review makes you 10x slower
News Roundup
The story of OpenBSD on Motorola 88000 series processors
+ jailrun github
FreeBSD Users: We Need to Talk About Claude Code
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 09 Apr 2026 - 1h 00min - 895 - 657: Hibernation is a long sleep
The Real Cost of Technology Dependence, FreeBSD 15 Linuxator with CUDA, Bidirectional OPNsense/pfSense, Netbase, a SYN attack, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The Real Cost of Technology Dependence: Building Independence with Open-Source Storage
News Roundup
Building Hierarchical Jails (Podman x Native Jail) on FreeBSD 15
FreeBSD 15.0 Linuxulator with CUDA Setup
Bidirectional OPNsense/pfSense Firewall Configuration Migration/Conversion CLI
Syn attack follow upNetbase is Port of NetBSD Utilities to Another UNIX Like Operating Systems
Beastie Bits
OpenBSD -current moves to 7.9-beta- Delayed hibernation comes to OpenBSD/amd64 laptops
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 02 Apr 2026 - 50min - 894 - 656: Honey, I shrunk the PDP
Designing OpenZFS Storage for Independence, The day Telnet died, PiDP 11/70, OpenBSD on SGI and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Designing OpenZFS Storage for Independence: Pool Architecture, Failure Domains, and Migration Paths
2026-01-14: The Day the telnet Died
Reports of Telnet’s Death Have Been Greatly ExaggeratedNews Roundup
OpenBSD on SGI: a rollercoaster story
Terminals Should Generate 256 Color Palette
FreeBSD tribal knowledge: Changes to snapshot strategy
Beastie Bits
BSDCan reg is now open An Oral History of Unix Major update to drm(4) code in OpenBSD-current (to linux 6.18.16) Patched FreeBSD AMIsTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 26 Mar 2026 - 1h 10min - 893 - 655: No Reboot Required
Jails for NetBSD, ARC and L2ARC sizing for Proxmox, Anatomy of bsd.rd, Docker Containers on FreeBSD, Running Time Machine inside a FreeBSD Jail, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
ARC and L2ARC Sizing on Proxmox
News Roundup
Lab: Anatomy of bsd.rd — No Reboot Required
Exploring Docker containers on FreeBSD
Time Machine inside a FreeBSD jail
After decades on Linux, FreeBSD finally gave me a reason to switch operating systems
Beastie Bits
-
-
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 19 Mar 2026 - 1h 00min - 892 - 654: Plasma Rage
Pool and Vdev topology for promox, KDE Plasma is not forcing systemd, Running a 2.11 BSD system, Booting NetBSD from a wedge and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Pool and VDEV Topology for Proxmox Workloads
News Roundup
KDE Plasma 6.6 is Not Forcing systemd(1) but Arguments Rage On.
An old article with covering : Running and administrating a 2.11 BSD system
Booting NetBSD from a wedge, the hard way
Beastie Bits
The NetBSD Foundation will participate in Google Summer of Code 2026! Solaris 11.4 SRU90: Preserve Boot Environments zfs-2.4.1 Hardening OPNsense: Using Q-Feeds to Block Malicious TrafficTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Gary - A nice blogSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 12 Mar 2026 - 45min - 891 - 653: Butter makes everything better
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
ZFS vs BTRFS Architects features and stability
RHEL on ZFS Root: An Unholy Experiment
News Roundup
Slackware on Encrypted ZFS Root.
https://tumfatig.net/2026/slackware-on-encrypted-zfs-root/OpenIndiana Is Porting Solaris' IPS Package Management To Rust
Tcl: The Most Underrated, But The Most Productive Programming Language
How to Setup WireGuard on OpenBSD: The Ultimate Self-Hosted VPN Guide (2026)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 05 Mar 2026 - 55min - 890 - 652: Ghostly Graphics
OpenZFS monitoring, hellosystems 0.8, GhostBSD and XLibre, Bhyve Exporters and 30 year old LibC issues.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
OpenZFS Monitoring and Observability: What to Track and Why It Matters
helloSystem 0.8 Released FreeBSD Based OS Inspired by macOS.
https://itsfoss.gitlab.io/post/hellosystem-08-released-freebsd-based-os-inspired-by-macos/News Roundup
[Default GhostBSD to XLibre](https://github.com/ghostbsd/ghostbsd-build/pull/259]
Addressing XLibre Change and GhostBSD FutureBhyve Prometheus Exporter for Sylve on FreeBSD.
Linux GNU C Library Fixes Security Issue Present Since 1996
Beastie Bits
NetBSD 11.0 RC1 available! The Book of PF, 4th Edition is now available December 2025 Finance Report LLDB improvements on FreeBSD Any desire for OnmiOS/Illumos Support : Now's your chance to convince meTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 26 Feb 2026 - 1h 10min - 889 - 651: Spatially aware ZFS
GeoIP PF FreeBSD, ZFs in production, linuxulator feels like magic, XFCE is great, the scariest boot code, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
GeoIP-Aware Firewalling with PF on FreeBSD
ZFS in Production: Real-World Deployment Patterns and Pitfalls
News Roundup
Linuxulator on FreeBSD Feels Like Magic
OpenBSD-current now runs as guest under Apple Hypervisor
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Matt - Audio LevelsInterviews can be troublesome because there's only so much we can do with multiple guests with multiple feeds, and mulitple audio conditions. We can try to normalize but sometimes it's just not easy to do without editing taking an entire day..
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 19 Feb 2026 - 57min - 888 - 650: Korn Chips
AT&T's $2000 shell, ZFS Scrubs and Data Integrity, FFS Backups, FreeBSD Home Nas, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
One too many words on AT&T's $2,000 Korn shell and other Usenet topics
Understanding ZFS Scrubs and Data Integrity
News Roundup
FreeBSD: Home NAS, part 1 – configuring ZFS mirror (RAID1)
8 more parts!Beastie Bits
The BSD Proposal UNIX Magic Poster Haiku OS Pulls In Updated Drivers From FreeBSD 15 FreeBSD 15.0 VNET Jails Call for NetBSD testingTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Gary - LinksSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 12 Feb 2026 - 57min - 887 - 649: The Desk Review
ZFS Scrubs and Data integrity, Propolice, FreeBSD vs Slackware and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Understanding ZFS Scrubs and Data Integrity
Desk reviews
describe comment ask questionsNo reponses, no justications.
[Tj's Desk](media/bsdnow649-tjs-desk.jpg)
[Ruben's Desk](media/bsdnow649-rubens-desk.jpg)News Roundup
FreeBSD vs. Slackware: Which super stable OS is right for you?
Prometheus, Let's Encrypt, and making sure all our TLS certificates are monitored
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 05 Feb 2026 - 1h 11min - 886 - 648: Greytrapping for years
FreeBSD's Future, 18 years of greytrapping, PF vs Linux firewalls, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Powering the Future of FreeBSD
Eighteen Years of Greytrapping - Is the Weirdness Finally Paying Off?
BSDCan Organisating committee Interview
News Roundup
How I, a non-developer, read the tutorial you, a developer, wrote for me, a beginner
BSD PF versus Linux nftables for firewalls for us
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 29 Jan 2026 - 1h 04min - 885 - 647: Why BSDs?
Why use BSD, 2025, the year of advocacy, community and growth, RiscV silicon, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
2025: A Year of Advocacy, Community, and Growth
What are we excited for in tech in 2026?
Almost Completley Open riscv32 silicon with BaoChip and Xous The death of vmware and the possibilites for diversity that creates Capabilities arriving in FreeBSD main NetBSD on the WiiU and open hardware for retro computing Capabilities arriving with CheriIOTNews Roundup
Beastie Bits
BastilleBSD 2025 User Survey Re: ZFS status on NetBSDTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Albin- Foss-NorthSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 22 Jan 2026 - 55min - 884 - 646: Unix v4
The Unix v4 recovery, webzfs, openbgpd 9.0, MidnightBSD 4.0, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
University of Utah team discovers rare computer relic
The attempt to read the UNIX V4 tape is underway!
UNIX V4 Tape from University of Utah
UNIX V4 tape successfully recovered: First ever version of UNIX written in C is running again
An initial analysis of the discovered Unix V4 tapeNews Roundup
Let's run FreeBSD 15.0-RELEASE on a Raspberry Pi Zero 2 W
Figuring out how I want to set up the TVPC
TVPC updateC&C Red Alert2 in your browser
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
rick - shout out.mdSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 15 Jan 2026 - 1h 14min - 883 - 645: Unwrapping gifts
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
What’s New in Oracle Solaris 11.4 SRU 87
News Roundup
Why I’m Trading My Linux for FreeBSD A Return to Simplicity
Our mixed assortment of DNS server software (as of December 2025)
Moving on...Thoughts, plans, and ideas...
FreeBSD + Flua
Json in Base
Building from SourceTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Dave - FreeBSD and FluaSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 08 Jan 2026 - 42min - 882 - 644: Holidays 2025 - What you been do'in?
Holidays 2025 - What you been do'in?
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
What tech did we enjoy playing with or found interesting in 2025?
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
- Gary - Storage Is Cheap
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 01 Jan 2026 - 1h 37min - 881 - 643: Unwrapping gifts
Upwrapping OpenZFS gifs, Propolice the OpenBSD Stack Protector, refreshing zpools, and the FreeBSD 15.0 release.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Unwrapping ZFS: Gifts from the Open Source Community
Who wins when we filter the open web through an opaque system?
News Roundup
We can't fund our way out of the free and open source maintenance problem
The story of Propolice, the OpenBSD stack protector
Copying everything off a zpool, destroying it, creating a new one, and copying everything back
All aboard the 15.0-RELEASE train!
Beastie Bits
Running A PDP-8 From 1965 The library of time OPNsense 25.7.9 released- OPNsense 25.10.1 business edition released
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Martin - recordingsSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 25 Dec 2025 - 1h 08min - 880 - 642: Look Harder
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Is DWPD Still a Useful SSD Spec?
Moving From Windows To FreeBSD As The Linux Chaos Alternative
Computer Chronicles Revisited 131 - Open Look, OSF/Motif, Macintosh IIcx and A/UX - Submitted by listener S.M. Oliva
News Roundup
We haven't seen ZFS checksum failures for a couple of years
Using FreeBSD to make self-hosting fun again
The usability of open source operating systems
The only existing copy of UNIX v4
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Is DWPD Still a Useful SSD Spec?
Moving From Windows To FreeBSD As The Linux Chaos Alternative
Computer Chronicles Revisited 131 - Open Look, OSF/Motif, Macintosh IIcx and A/UX - Submitted by listener S.M. Oliva
News Roundup
We haven't seen ZFS checksum failures for a couple of years
Using FreeBSD to make self-hosting fun again
The usability of open source operating systems
The only existing copy of UNIX v4
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 18 Dec 2025 - 1h 17min - 879 - 641: Open to Free
FreeBSD 15 release, moving from OpenBSD to FreeBSD, ZFS Boot Environments explained, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Welcome to the world FreeBSD 15.0-RELEASE Announcement and Release Notes
We're (now) moving from OpenBSD to FreeBSD for Firewalls - Submitted by listener Gary
News Roundup
ZFS Boot Environments Explained
rocinante - A configuration management tool by the BastilleBSD team
A Grown-up ZFS Data Corruption Bug and YouTube
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Claudio - A Silent ReflectionSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 11 Dec 2025 - 55min - 878 - 640: Cleaning up Hammer
FreeBSD is an OCI runtime, ZFS Disaster Recovery, Cleaning up Hammer, and some historical information, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
FreeBSD Officially Supported in OCI Runtime Specification v1.3
ZFS Enabled Disaster Recovery for Virtualization
News Roundup
How I think OpenZFS's 'written' and 'written@' dataset properties work
Make sure your Hammer cleanup cleans up
[TUHS] David C Brock of CHM: 2024 oral history with Ken Thompson + Doug McIlroy
Special Issue “Celebrating 60 Years of ELIZA? Critical Pasts and Futures of AI”
Source and state limiters introduced in pf
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Göran - grafanaSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 04 Dec 2025 - 36min - 877 - 639: Reproducible Builds
Reproducible builds, Highly available ZFS Pools, Self Hosting on a Framework Laptop, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
FreeBSD now builds reproducibly and without root privilege
How to Set Up a Highly Available ZFS Pool Using Mirroring and iSCSI
News Roundup
Self hosting 10TB in S3 on a framework laptop + disks
Some notes on OpenZFS's 'written' dataset property
Big news for small /usr partitions
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Patrick - FeedbackSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 27 Nov 2025 - 1h 00min - 876 - 638: Hipsters want their distribution back
New Open Indiana Release, Understanding Storage Performance, a Unix OS for the TI99, FreeBSD Tribal knowledge, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Open Indiana Hipster Announcement
Understanding Storage Performance Metrics
News Roundup
UNIX99, a UNIX-like OS for the TI-99/4A
Making the veb(4) virtual Ethernet bridge VLAN aware
FreeBSD tribal knowledge: minor version upgrades
It's been 10 years since ZFS's 10th aniversary its integration into Solaris - A Reflection
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Sponsored By:
Tarsnap Promo Code: bsdnowThu, 20 Nov 2025 - 1h 08min - 875 - 637: /etc/hosts
Time to update our /etc/hosts file...
NOTES*
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Show updates
Intro Ruben
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 13 Nov 2025 - 1h 07min - 874 - 636: Thunder Bolts
Thunderbolt on FreeBSD, ZFS on Illumos and Linux and FreeBSD, ZFS Compression, Home networking monitoring, LibreSSH and OpenSSH releases and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The broad state of ZFS on Illumos, Linux, and FreeBSD (as I understand it)
News Roundup
zfs: setting compression and adding new vdevs
The hunt for a home network monitoring solution
Related to 10.x versions : Post-Quantum CryptographyCheck your IP infos using nginx
Experimenting with Compression
(just given an overview, I dont exepect you to read the all three writeups fully)
Experimenting with compression off Experimenting with compression=lz4 Experimenting with compression=zstd Compression resultsTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Anton - BoxybsdSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 06 Nov 2025 - 1h 03min - 873 - 635: Guess who's back?
OpenBSD 7.8, Building Enterprise Storage with Proxmox, SSD performance, Virtual Machines and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
OpenBSD 7.8 Released also (https://undeadly.org/cgi?action=article;sid=20251022025822) and (https://bsd.network/@brynet/115403567146395679)
Building Enterprise-Grade Storage on Proxmox with ZFS
News Roundup
[TUHS] Was artifacts, now ethernet
I wish SSDs gave you CPU performance style metrics about their activity
Migrate a KVM virtual machine to OmniOS bhyve
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
brad - bhyveSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 30 Oct 2025 - 1h 17min - 872 - 634: Why Self-Host?
Why Self-host?, Advanced ZFS Dataset Management, Building a Simple Router with OpenBSD, Minimal pkgbase jails / chroots, WSL-For-FreeBSD, Yubico yubikey 5 nfc on FreeBSD, The Q3 2025 Issue of the FreeBSD Journal, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Advanced ZFS Dataset Management: Snapshots, Clones, and Bookmarks
News Roundup
Building a Simple Router with OpenBSD
Minimal pkgbase jails / chroots
Yubico yubikey 5 nfc on FreeBSD
The Q3 2025 Issue of the FreeBSD Journal is Now Available
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 23 Oct 2025 - 1h 01min - 871 - 633: Magical Systems Thinking
ZFS Features, Roadmap, and Innovations, Magical systems thinking, How VMware’s Debt-Fueled Acquisition Is Killing Open Source, OpenSSH 10.1 Released, KDE Plasma 6 Wayland on FreeBSD, Unix Co-Creator Brian Kernighan on Rust, Distros and NixOS, Balkanization of the Internet, GhostBSD 25.02 adds 'Gershwin' desktop for a Mac-like twist, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
What the Future Brings – ZFS Features, Roadmap, and Innovations
News Roundup
KDE Plasma 6 Wayland on FreeBSD
Unix Co-Creator Brian Kernighan on Rust, Distros and NixOS
GhostBSD 25.02 adds 'Gershwin' desktop for a Mac-like twist
Beastie Bits
Adventures in porting a Wayland Compositor to NetBSD and OpenBSD by Jeff FrascaTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Kylen - CVEsSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 16 Oct 2025 - 1h 06min - 870 - 632: Zipbomb defeated
zipbomb defeated, Optimizing ZFS for High-Throughput Storage Workloads, Open Source is one person, Omada SDN Controller on FreeBSD, Building a Simple Router with OpenBSD, Back to the origins, Enhancing Support for NAT64 Protocol Translation in NetBSD, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Optimizing ZFS for High-Throughput Storage Workloads
News Roundup
Omada SDN Controller on FreeBSD
Google Summer of Code 2025 Reports: Enhancing Support for NAT64 Protocol Translation in NetBSD
Undeadly Bits
j2k25 - OpenBSD Hackathon Japan 2025 OpenSSH will now adapt IP QoS to actual sessions and traffic Preliminary support for Raspberry Pi 5 OpenBSD enters 7.8-beta Full BSDCan 2025 video playlist(s) available OpenBGPD 8.9 releasedTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Brad - a few thingsSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 09 Oct 2025 - 52min - 869 - 630: Bhyve Management UI
FreeBSD Foundation Q2 2025 Status Update, Keeping Data Safe with OpenZFS, Ollama on FreeBSD Using GPU Passthrough, ClonOS, Preliminary support for Raspberry Pi 5, Sylve: Manage bhyve VMs and Clusters on FreeBSD, Preventing Systemd DHCP RELEASE Behavior, Call for testing - Samba 4.22, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
FreeBSD Foundation Q2 2025 Status Update
Keeping Data Safe with OpenZFS: Security, Encryption, and Delegation
News Roundup
Ollama on FreeBSD Using GPU Passthrough
Preliminary support for Raspberry Pi 5
Sylve: Manage bhyve VMs and Clusters on FreeBSD
Preventing Systemd DHCP RELEASE Behavior
Call for testing - Samba 4.22 in 0mp's ports tree
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 02 Oct 2025 - 35min - 868 - 631: Endorphin Rush
Secure Boot for FreeBSD, Systems lie about their proper functioning, Teching the tech and rushing the endorphins, Passing a Device Into A FreeBSD Jail With A Stable Name, ZFS snapshots aren't as immutable as I thought, due to snapshot metadata, Let's write a peephole optimizer for QBE's arm64 backend, Migrate a Peertube instance from Debian to FreeBSD, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The Fundamental Failure-Mode Theorem: Systems lie about their proper functioning
News Roundup
Teching the tech and rushing the endorphins
Passing a Device Into A FreeBSD Jail With A Stable Name
ZFS snapshots aren't as immutable as I thought, due to snapshot metadata
Let's write a peephole optimizer for QBE's arm64 backend
Migrate a Peertube instance from Debian to FreeBSD
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 25 Sep 2025 - 36min - 867 - 629: Host Naming Conventions
The Death of Industrial Design, Host naming Convensions, Symbian reflections, bash timeouts, nvme vs ssds, a system to organize your life, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The Death Of Industrial Design And The Era Of Dull Electronics
News Roundup
Open, free, and completely ignored: The strange afterlife of Symbian
It seems like NVMe SSDs have overtaken SATA SSDs for high capacities
A system to organise your life
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
- Nelson - Books
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 18 Sep 2025 - 1h 08min - 866 - 628: Product Hype
The Hype is the Product, Programmers Aren’t So Humble Anymore—Maybe Because Nobody Codes in Perl, Is OpenBSD 10x faster than Linux?, How to install FreeBSD on providers that don't support it with mfsBSD, SSHX, Zvault Status Update, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Programmers Aren’t So Humble Anymore—Maybe Because Nobody Codes in Perl
News Roundup
Is OpenBSD 10x faster than Linux?
How to install FreeBSD on providers that don't support it with mfsBSD
Undeadly Bits
4096 colours and flashing text on the console! Font caching no longer runs as root OpenSSH will now adapt IP QoS to actual sessions and trafficTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 11 Sep 2025 - 49min - 865 - 627: Catastrophic OpenZFS bug
An (almost) catastrophic OpenZFS bug, crawler plague and the fragility of the web, Classic CDE (Common Desktop Environment) coming to OpenBSD, Some notes on DMARC policy inheritance and a gotcha, GNAT (Ada) is in fact fully supported on illumos, Eighteen Years of Greytrapping, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
An (almost) catastrophic OpenZFS bug and the humans that made it (and Rust is here too)
The current (2025) crawler plague and the fragility of the web
News Roundup
Classic CDE (Common Desktop Environment) coming to OpenBSD
Some notes on DMARC policy inheritance and a gotcha
Despite thoughts to the contrary, GNAT (Ada) is in fact fully supported on illumos
Eighteen Years of Greytrapping - Is the Weirdness Finally Paying Off?
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 04 Sep 2025 - 55min - 864 - 626: USB webcam testing
FreeBSD Journal Summer 2025 Edition, Java hiding in plain sight, BSDCan 2025 Trip report, Call for testing OpenBSD webcams, recent new features in OpenSSH, Improved 802.11g AP compatibility check, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
FreeBSD Journal April/May/June 2025 Edition
BSDCan 2025 Trip Report – Chuck Tuffli
News Roundup
From Minecraft to Markets: Java Hiding in Plain Sight
Recent new features in OpenSSH
NetBSD 11.0 release process underway
Interview: Nico Cartron
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Special Guest: Nico Cartron.
Thu, 28 Aug 2025 - 56min - 863 - 625: Build Cluster Speedup
Why FreeBSD is the Right Choice for Embedded Devices, The Day GlusterFS Tried to Kill My Career, DragonFly DRM updated, NetBSD on Raspberry Pi, Speed up suspend/resume for FreeBSD, Revisiting ZFS's ZIL, separate log devices, and writes, One of my blog articles featured on the BSD Now podcast episode, New build cluster speeds up daily autobuilds, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Why FreeBSD is the Right Choice for Embedded Devices
The Day GlusterFS Tried to Kill My Career
News Roundup
Speed up suspend/resume for FreeBSD
Revisiting ZFS's ZIL, separate log devices, and writes
One of my blog articles featured on the BSD Now podcast episode!
New build cluster speeds up daily autobuilds
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 21 Aug 2025 - 50min - 862 - 624: OpenBSD Innovations
OpenBSD chflags vs. Log Tampering, How to Defend Against Aggressive Web Scrapers With Anubis on FreeBSD 14, OpenBSD Innovations, Full Ada programming toolchain NOW on FreeBSD, Compute GPUs can have odd failures under Linux (still), A handy collection of shell aliases from my bash startup, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
When Root Meets Immutable: OpenBSD chflags vs. Log Tampering
How to Defend Against Aggressive Web Scrapers With Anubis on FreeBSD 14
News Roundup
Full Ada programming toolchain NOW on FreeBSD
Compute GPUs can have odd failures under Linux (still)
A handy collection of shell aliases from my bash startup
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Efraim - modernizingSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 14 Aug 2025 - 1h 01min - 861 - 623: Two's interview
Software Bill of Materials (SBOM) for FreeBSD Project, Your Guide to Lock-In Free Infrastructure, and we interview David Gwynne from the University of Queensland and developer on the OpenBSD project.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Software Bill of Materials (SBOM) for FreeBSD Project
FreeBSD Summer 2025 Roundup: Your Guide to Lock-In Free Infrastructure
Interview
David Gwynne from the University of Queensland and developer on the OpenBSD project.Interview thoughts from Benedict and Jason
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Special Guest: David Gwynne.
Thu, 07 Aug 2025 - 1h 00min - 860 - 622: Interview with Mark Phillips - Technical Marketing Manager at the FreeBSD Foundation
This week Benedict interviews Mark Phillips , the Technical Marketing Manager at the FreeBSD Foundation, while they both are at a Hackathon in Germany.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Interview
Mark Phillips - Technical Marketing Manager at the FreeBSD Foundation
Personal websiteTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Special Guest: Mark Phillips.
Thu, 31 Jul 2025 - 55min - 859 - 621: Exaggerated Death Report
Designing a Storage Pool, The Report of My Death Was an Exaggeration, Generic BSD installations on ARM64 UEFI, dm_target_crypt_ng - Add next-generation implementation, The X Window System didn't immediately have X terminals, The Book of PF 4th Edition Is Coming Soon, Periodical 20 Localized Computing, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Designing a Storage Pool: RAIDZ, Mirrors, and Hybrid Configurations
The Report of My Death Was an Exaggeration
News Roundup
Generic BSD installations on ARM64 UEFI: results and first impressions
dm_target_crypt_ng - Add next-generation implementation
The X Window System didn't immediately have X terminals
Yes, The Book of PF, 4th Edition Is Coming Soon
Periodical 20 — Localized Computing
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 24 Jul 2025 - 50min - 858 - 620: Postmortem for jemalloc
The Server That Wasn't Meant to Exist, ZFS Performance Tuning – Optimizing for your Workload, what would a multi-user web server look like, That Grumpy BSD Guy: A Short Reading List, rsync's defaults are not always enough, jemalloc Postmortem, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The Server That Wasn't Meant to Exist
ZFS Performance Tuning – Optimizing for your Workload
News Roundup
What would a multi-user web server look like? (A thought experiment)
That Grumpy BSD Guy: A Short Reading List
rsync's defaults are not always enough
Beastie Bits
IPv6 and proxying on DragonFly BoxyBSD SysctltuiTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 17 Jul 2025 - 53min - 857 - 619: Happy Tooling
Disaster Recovery with ZFS: A Practical Guide, The best interfaces we never built, Choose Tools That Make You Happy, open source has turned into two worlds, TrueNAS CORE is Dead – Long Live zVault, You should start a computer club in the place that you live, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Disaster Recovery with ZFS: A Practical Guide
The best interfaces we never built
News Roundup
You Can Choose Tools That Make You Happy
I feel open source has turned into two worlds
UPDATE 2 – TrueNAS CORE is Dead – Long Live zVault
You should start a computer club in the place that you live
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Brad - syslogng issueSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 10 Jul 2025 - 45min - 856 - 618: Funding BSD projects
A year of funded FreeBSD, ZFS Performance Tuning – Optimizing for your Workload, Three Ways to Try FreeBSD in Under Five Minutes, FFS optimizations with dirhash, j2k25 hackathon report from kn@, NetBSD welcomes Google Summer of Code contributors, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
ZFS Performance Tuning – Optimizing for your Workload
News Roundup
Three Ways to Try FreeBSD in Under Five Minutes
FFS optimizations with dirhash
j2k25 hackathon report from kn@: installer, low battery, and more
NetBSD welcomes Google Summer of Code contributors
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 03 Jul 2025 - 53min - 855 - 617: FreeBSD 14.3
FreeBSD version 14.3 is available, Reliable ZFS Storage on Commodity Hardware, My website is ugly because I made it, Semi distributed filesystems with ZFS and Sanoid, April 2025 Laptop Support and Usability Project Update, UDP sockets instead of BPF in dhcpd(8), and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Reliable ZFS Storage on Commodity Hardware
News Roundup
My website is ugly because I made it
Semi distributed filesystems with ZFS and Sanoid
April 2025 Laptop Support and Usability Project Update
dhcpd(8): use UDP sockets instead of BPF
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
No feedback this week. Send more...
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 26 Jun 2025 - 1h 03min - 854 - 616: FreeBSD Foundation Interview
This week on the show Tom interview Deb Goodkin and Justin Gibbs from the FreeBSD Foundation.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Guests
Deb Goodkin Justin GibbsSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Special Guests: Deb Goodkin and Justin Gibbs.
Thu, 19 Jun 2025 - 47min - 853 - 615: Wifi Brakes Unlocked
How to unlock high speed Wi-Fi on FreeBSD 14, What We’ve Learned Supporting FreeBSD in Production, rsync replaced with openrsync on macOS Sequoia, Framework 13 AMD Setup with FreeBSD, FreeBSD on Dell Latitude 7280, Backup MX with OpenSMTPD, Notes on caddy as QUIC reverse proxy with mac_portacl, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
How to unlock high speed Wi-Fi on FreeBSD
14What We’ve Learned Supporting FreeBSD in Production
News Roundup
rsync replaced with openrsync on macOS Sequoia
Framework 13 AMD Setup with FreeBSD
Notes on caddy as QUIC reverse proxy with mac_portacl
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
No feedback this week.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 12 Jun 2025 - 44min - 852 - 614: Upstream Contributions Matter
The Hidden Costs of Stagnation: Why Running EOL Software is a Ticking Time Bomb, Maintaining FreeBSD in a Commercial Product – Why Upstream Contributions Matter, LLMs ('AI') are coming for our jobs whether or not they work, Implement Anubis to give the bots a harder time, erspan(4): ERSPAN Type II collection, Just my memory here is how I've configure OpenBSD and FreeBSD for a IPv6 Wifi, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The Hidden Costs of Stagnation: Why Running EOL Software is a Ticking Time Bomb
Maintaining FreeBSD in a Commercial Product – Why Upstream Contributions
MatterNews Roundup
LLMs ('AI') are coming for our jobs whether or not they work
Implement Anubis to give the bots a harder time
erspan(4): ERSPAN Type II collection
Just my memory here is how I've configure OpenBSD and FreeBSD for a IPv6 Wifi
Beastie Bits
Some Interesting pieces of history
Netnews History History of Solaris Nuclear Wall Charts [TUHS] The Case of UNIX vs. The UNIX SystemTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Paul - my setupSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 05 Jun 2025 - 1h 03min - 851 - 613: DragonflyBSD 6.4.2
Isolating Containers with ZFS and Linux Namespaces, DragonFly BSD 6.4.2, FreeBSD fans rally round zVault upstart, For Upcoming PF Tutorials, We Welcome Your Questions, Using ~/.ssh/authorized keys to decide what the incoming connection can do, PDF bruteforce tool to recover locked files, How and why typical (SaaS) pricing is too high for university departments, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Isolating Containers with ZFS and Linux Namespaces
FreeBSD fans rally round zVault upstart
News Roundup
For Upcoming PF Tutorials, We Welcome Your Questions
Using ~/.ssh/authorized keys to decide what the incoming connection can do
PDF bruteforce tool to recover locked files
How and why typical (SaaS) pricing is too high for university departments
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Nils - CFPSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 29 May 2025 - 53min - 850 - 612: Zip Bomb Protection
I use Zip Bombs to Protect my Server, Owning the Stack: Infrastructure Independence with FreeBSD and ZFS, Optimisation of parallel TCP input, Chosing between "it works for now" and "it works in the long term", Losing one of my evenings after an OpenBSD upgrade, What drive did I just remove from the system?, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
I use Zip Bombs to Protect my Server
Owning the Stack: Infrastructure Independence with FreeBSD and ZFS
News Roundup
Optimisation of parallel TCP input
Chosing between "it works for now" and "it works in the long term"
Losing one of my evenings after an OpenBSD upgrade
What drive did I just remove from the system?
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Benjamin - Street PCsSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 22 May 2025 - 37min - 849 - 611: Ghosty Things
GhostBSD: From Usability to Struggle and Renewal, Why You Can’t Trust AI to Tune ZFS, Introducing bpflogd(8): capture packets via BPF to log files, What I'd do as a College Freshman in 2025, FreeBSD and KDE Plasma generations, Improvements to the FreeBSD CI/CD systems, FreeBSD as a Workstation, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
GhostBSD: From Usability to Struggle and Renewal
Why You Can’t Trust AI to Tune ZFS
News Roundup
Introducing bpflogd(8): capture packets via BPF to log files
What I'd do as a College Freshman in 2025
FreeBSD and KDE Plasma generations
Improvements to the FreeBSD CI/CD systems
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Effie - FreeBSD as a Workstation
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 15 May 2025 - 49min - 848 - 610: OpenBSD 7.7
OpenBSD 7.7, ZFS Orchestration Tools – Part 2: Replication, Switching customers from Linux to BSD because boring is good, Graphed and measured: running TCP input in parallel, Introducing an OpenBSD LLDP daemon, Hardware discovery: ACPI & Device Tree, The 2025 FreeBSD Community Survey is Here, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
ZFS Orchestration Tools – Part 2: Replication
News Roundup
Switching customers from Linux to BSD because boring is good
Graphed and measured: running TCP input in parallel
Introducing an OpenBSD LLDP daemon
Hardware discovery: ACPI & Device Tree
The 2025 FreeBSD Community Survey is Here
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Brad - new usersSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 08 May 2025 - 1h 02min - 847 - 609: Toe-Dipping in Amsterdam
Inside FreeBSD Netgraph: Behind the Curtain of Advanced Networking, Launching BSSG - My Journey from Dynamic CMS to Bash Static Site Generator, OpenZFS Cheat Sheet, Dipping my toes in OpenBSD in Amsterdam, SSH keys from a command: sshd's AuthorizedKeysCommand directive, How to move bhyve VM and Jail container from one host to another host, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
Inside FreeBSD Netgraph: Behind the Curtain of Advanced Networking
Launching BSSG - My Journey from Dynamic CMS to Bash Static Site Generator
News Roundup
Dipping my toes in OpenBSD, in Amsterdam
SSH keys from a command: sshd's AuthorizedKeysCommand directive
How to move bhyve VM and Jail container from one host to another host ?
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Dave - WebstackSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 01 May 2025 - 54min - 846 - 608: Reboot required
Robust & Reliable Backup Solutions with OpenZFS, Why I Maintain a 17 Year Old Thinkpad, Motivations, Tinker Writer Deck, How to tell if FreeBSD needs a Reboot using kernel version check, Techie pulled an all-nighter that one mistake turned into an all-weekender, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
World Backup Day 2025: Robust & Reliable Backup Solutions with OpenZFS
Why I Maintain a 17 Year Old Thinkpad
News Roundup
How to tell if FreeBSD needs a Reboot using kernel version check
Techie pulled an all-nighter that one mistake turned into an all-weekender
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Ian - Personal Web Stack Brendan - Storage BackendsSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 24 Apr 2025 - 48min - 845 - 607: Sign those commits
We should improve libzfs somewhat, Accurate Effective Storage Performance Benchmark, Debugging aids for pf firewall rules on FreeBSD, OpenBSD and Thunderbolt issue on ThinkPad T480s, Signing Git Commits with an SSH key, Pgrep, LibreOffice downloads on the rise, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
We should improve libzfs somewhat
Accurate Effective Storage Performance Benchmark
News Roundup
Debugging aids for pf firewall rules on FreeBSD
OpenBSD and Thunderbolt issue on ThinkPad T480s
Signing Git Commits with an SSH key
LibreOffice downloads on the rise as users look to avoid subscription costs
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Felix - Bhyve and NVMESend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 17 Apr 2025 - 56min - 844 - 606: Tackling 7k bugs
FreeBSD 13.5-RELEASE Now Available, From Chaos to Clarity: How We Tackled FreeBSD’s 7,000 Bug Backlog, zfs-2.3.1, Complications of funding an open source operating system, Why Choose to Use the BSDs in 2025, First Use on GhostBSD, Better Shell History Search, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
FreeBSD 13.5-RELEASE Now Available
From Chaos to Clarity: How We Tackled FreeBSD’s 7,000 Bug Backlog
News Roundup
Complications of funding an open source operating system
Why Choose to Use the BSDs in 2025
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Russell - QuestionsSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 10 Apr 2025 - 1h 11min - 843 - 605: Fediverse Weather Service
FediMeteo: How a Tiny €4 FreeBSD VPS Became a Global Weather Service for Thousands, Core Infrastructure: Why You Need to Control Your NTP, Automatic Display switch for OpenBSD laptop, Using a 2013 Mac Pro as a FreeBSD Desktop, Some terminal frustrations, Copying all files of a directory, including hidden ones, with cp, You Should Use /tmp/ More, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
FediMeteo: How a Tiny €4 FreeBSD VPS Became a Global Weather Service for Thousands
Core Infrastructure: Why You Need to Control Your NTP
News Roundup
Automatic Display switch for OpenBSD laptop
Using a 2013 Mac Pro as a FreeBSD Desktop
Copying all files of a directory, including hidden ones, with cp
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Tyler - Toms requestSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 03 Apr 2025 - 58min - 842 - 604: Future looks back
The Future Looking Back At Us: Joanne McNeil on Cyberpunk, Why ZFS reports less available space, We are destroying software, FreeBSD 13.5 Overcomes UFS Y2038 Problem To Push It Out To Year 2106, 1972 UNIX V2 "Beta" Resurrected, Some thoughts on why 'inetd activation' didn't catch on, If you believe in “Artificial Intelligence”, take five minutes to ask it about stuff you know well, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The Future Looking Back At Us: Joanne McNeil on Cyberpunk
Why ZFS reports less available space space accounting explained/
News Roundup
FreeBSD 13.5 Overcomes UFS Y2038 Problem To Push It Out To Year 2106
TUHS: 1972 UNIX V2 "Beta" Resurrected
Some thoughts on why 'inetd activation' didn't catch on
If you believe in “Artificial Intelligence”, take five minutes to ask it about stuff you know well
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Nelson - gcc puzzlementSend questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 27 Mar 2025 - 49min - 841 - 603: Expanding the RAID-Z
OpenZFS RAID-Z Expansion: A New Era in Storage Flexibility, ZFS Orchestration Tools – Part 1: Snapshots, The Case of UNIX vs. The UNIX System, OpenBGPD 8.8 released, OPNsense 25.1, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
OpenZFS RAID-Z Expansion: A New Era in Storage Flexibility
ZFS Orchestration Tools – Part 1: Snapshots
News Roundup
Manage OpenBSD with AWS Systems Manager
TUHS:The Case of UNIX vs. The UNIX System
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 20 Mar 2025 - 36min - 840 - 602: Wildcard Gotchas
I Tried FreeBSD as a Desktop in 2025. Here's How It Went, Cray 1 Supercomputer Performance Comparisons With Home Computers Phones and Tablets, The first perfect computer, Find Name Wildcard Gotcha, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
I Tried FreeBSD as a Desktop in 2025. Here's How It Went
Cray 1 Supercomputer Performance Comparisons With Home Computers Phones and Tablets
News Roundup
State of virtualizing the BSDs on Apple Silicon
New Patreon Levels
Level 1 - user memory (Tip Jar) @ $1 / month
Show your support for the showLevel 2 - virtual memory (Ad-Free Episodes) @ $5 / month
Ad-free episodesLevel 3 - kmem (VIP Patron) @ $10 / month
Everything in higher memory levels &
Your feedback and questions jump the queue and go in the next episode.
Personal shout outs (with your consent) for recommending articles we cover.Level 4 - physical memory @ $20 / month
What's included:
Everything in higher memory levels &
You can send in audio/video questions and we'll air your audio in the show feedback section (if the quality of your recording is decent)
Behind-the-scenes content - Raw Video from Recording sessions with intro/outro discussion not included in the show
Additional Content when we all make itTarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 13 Mar 2025 - 57min - 839 - 601: The Monospace Web
The PC is Dead: It’s Time to Make Computing Personal Again, The Biggest Unix Security Loophole, The monospace Web, What a FreeBSD kernel message about your bridge means, Installing FreeBSD on a HP 250 G9, Networking for System Administrators, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap and the BSDNow Patreon
Headlines
The PC is Dead: It’s Time to Make Computing Personal Again
The Biggest Unix Security Loophole
News Roundup
What a FreeBSD kernel message about your bridge means
Installing FreeBSD on a HP 250 G9
Networking for System Administrators
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
Join us and other BSD Fans in our BSD Now Telegram channel
Thu, 06 Mar 2025 - 47min - 838 - 238: VLAN-Zezes-ki in Hardware
Looking at Lumina Desktop 2.0, 2 months of KPTI development in SmartOS, OpenBSD email service, an interview with Ryan Zezeski, NomadBSD released, and John Carmack's programming retreat with OpenBSD.
This episode was brought to you by
Headlines
Looking at Lumina Desktop 2.0
Q: Are there future plans to implement something like Lumina in a MAC Jail?A few weeks ago I sat down with Lead Developer Ken Moore of the TrueOS Project to get answers to some of the most frequently asked questions about Lumina Desktop from the open source community. Here is what he said on Lumina Desktop 2.0. Do you have a question for Ken and the rest of the team over at the TrueOS Project? Make sure to read the interview and comment below. We are glad to answer your questions!
Ken: Lumina Desktop 2.0 is a significant overhaul compared to Lumina 1.x. Almost every single subsystem of the desktop has been streamlined, resulting in a nearly-total conversion in many important areas.
With Lumina Desktop 2.0 we will finally achieve our long-term goal of turning Lumina into a complete, end-to-end management system for the graphical session and removing all the current runtime dependencies from Lumina 1.x (Fluxbox, xscreensaver, compton/xcompmgr). The functionality from those utilities is now provided by Lumina Desktop itself.
Going along with the session management changes, we have compressed the entire desktop into a single, multi-threaded binary. This means that if any rogue script or tool starts trying to muck about with the memory used by the desktop (probably even more relevant now than when we started working on this), the entire desktop session will close/crash rather than allowing targeted application crashes to bypass the session security mechanisms. By the same token, this also prevents man-in-the-middle type of attacks because the desktop does not use any sort of external messaging system to communicate (looking at you
dbus). This also gives a large performance boost to Lumina DesktopThe entire system for how a users settings get saved and loaded has been completely redone, making it a layered settings system which allows the default settings (Lumina) to get transparently replaced by system settings (OS/Distributor/SysAdmin) which can get replaced by individual user settings. This results in the actual changes in the user setting files to be kept to a minimum and allows for a smooth transition between updates to the OS or Desktop. This also provides the ability to restrict a users desktop session (based on a system config file) to the default system settings and read-only user sessions for certain business applications.
The entire graphical interface has been written in QML in order to fully-utilize hardware-based GPU acceleration with OpenGL while the backend logic and management systems are still written entirely in C++. This results in blazing fast performance on the backend systems (myriad multi-threaded C++ objects) as well as a smooth and responsive graphical interface with all the bells and whistles (drag and drop, compositing, shading, etc).
Q: I look forward to these changes. I know the last time I used it when I would scroll I would get flashes like the refresh rate was not high enough. It will be nice to have a fast system as well as I know with the more changes Linux is becoming slower. Not once it has loaded but in the loading process. I will do another download when these changes come out and install again and maybe stay this time.While I have never tried out Lumina in a MAC jail, I do not see anything on that page which should stop it from running in one right now. Lumina is already designed to be run as an unpriviledged user and is very smart about probing the system to find out what is/not available before showing anything to the user. The only thing that comes to mind is that you might need to open up some other system devices so that X11 itself can draw to the display (graphical environment setup is a bit different than CLI environment).
Q: Why was enlightenment project not considered for TrueOS? It is BSD licensed and is written in C.If I recall correctly, one of the very first versions of Lumina (pre-1.0) would occasionally flicker. If that is still happening, you might want to verify that you are using the proper video driver for your hardware and/or enable the compositor within the Lumina settings.
Q: I have two questions. 1) The default layout of Unity(menu bar with actual menu entries on top and icon dock on the side) is one of the few things I liked about my first voyage into non-Windows systems, and have been missing since moving on to other distros(and now also other non-Linux systems). However in 1.4.0 screenshots on Luminas site, the OSX-like layout has the menu attached to the window. Will 2.0 be able to have the menus on the bar? 2) Is there any timeline for a public release, or are you taking a when its ready approach?This was a common question about 4(?) years ago with the first release of the Lumina desktop and it basically boiled down to long-term support and reliability of the underlying toolkit. Some of the things we had to consider were: cross-platform/cross-architecture support, dependency reliability and support framework (Qt5 > EFL), and runtime requirements and dependency tracking (Qt5 is lighter than the EFL). That plus the fact that the EFL specifically states that it is linux-focused and the BSDs are just an afterthought (especially at the time we were doing the evaluation).
- In Lumina you can already put panels on the left/right side of the screen and give you something like the layout of the Unity desktop. The embedded menu system is not available in Lumina because that is not a specification supported by X11 and the window manager standards at the present time. The way that functionality is currently run on Linux is a hacky-bypass of the display system which only really works with the GTK3 and Qt5 toolkits, resulting in very odd overall desktop behavior in mixed environments where some apps use other graphical toolkits. We are targetting the 18.06 STABLE release of TrueOS for Lumina 2, but that is just a guideline and if necessary we will push back the release date to allow for additional testing/fixing as needed.
A long two months
IllumOS/SmartOS developer Alex Wilson describes the journey of developing KPTI for IllumOS > On Monday (January 1st) I had the day off work for New Year's day, as is usual in most of the western world, so I slept in late. Lou and her friend decided to go to the wax museum and see several tourist attractions around SF, and I decided to pass the day at home reading. That afternoon, work chat started talking about a Tumblr post by pythonsweetness about an Intel hardware security bug. At the time I definitely did not suspect that this was going to occupy most of my working life for the next (almost) two months.Like many people who work on system security, I had read Anders Fogh's post about a "Negative Result" in speculative execution research in July of 2017. At the time I thought it was an interesting writeup and I remember being glad that researchers were looking into this area. I sent the post to Bryan and asked him about his thoughts on it at the time, to which he replied saying that "it would be shocking if they left a way to directly leak out memory in the speculative execution". None of us seriously thought that there would be low-hanging fruit down that research path, but we also felt it was important that there was someone doing work in the area who was committed to public disclosure.
At first, after reading the blog post on Monday, we thought (or hoped) that the bug might "just" be a KASLR bypass and wouldn't require a lot of urgency. We tried to reach out to Intel at work to get more information but were met with silence. (We wouldn't hear back from them until after the disclosure was already made public.) The speculation on Tuesday intensified, until finally on Wednesday morning I arrived at the office to find links to late Tuesday night tweets revealing exploits that allowed arbitrary kernel memory reads.
Wednesday was not a happy day. Intel finally responded to our emails -- after they had already initiated public disclosure. We all spent a lot of time reading. An arbitrary kernel memory read (an info leak) is not that uncommon as far as bugs go, but for the most part they tend to be fairly easy to fix. The thing that makes the Meltdown and Spectre bugs particularly notable is that in order to mitigate them, a large amount of change is required in very deep low-level parts of the kernel. The kind of deep parts of the kernel where there are 20-year old errata workarounds that were single-line changes that you have to be very careful to not accidentally undo; the kind of parts where, as they say, mortals fear to tread.
On Friday we saw the patches Matthew Dillon put together for DragonFlyBSD for the first time. These were the first patches for KPTI that were very straightforward to read and understand, and applied to a BSD-derived kernel that was similar to those I'm accustomed to working on.
To mitigate Meltdown (and partially one of the Spectre variants), you have to make sure that speculative execution cannot reach any sensitive data from a user context. This basically means that the pages the kernel uses for anything potentially sensitive have to be unmapped when we are running user code. Traditionally, CPUs that were built to run a multi-user, UNIX-like OS did this by default (SPARC is an example of such a CPU which has completely separate address spaces for the kernel and userland). However, x86 descends from a single-address-space microcontroller that has grown up avoiding backwards-incompatible changes, and has never really introduced a clean notion of multiple address spaces (segmentation is the closest feature really, and it was thrown out for 64-bit AMD64). Instead, operating systems for x86 have generally wound up (at least in the post-AMD64 era) with flat address space models where the kernel text and data is always present in the page table no matter whether you're in user or kernel mode. The kernel mappings simply have the "supervisor" bit set on them so that user code can't directly access them.
The mitigation is basically to stop doing this: to stop mapping the kernel text, data and other memory into the page table while we're running in userland. Unfortunately, the x86 design does not make this easy. In order to be able to take interrupts or traps, the CPU has to have a number of structures mapped in the current page table at all times. There is also no ability to tell an x86 CPU that you want it to switch page tables when an interrupt occurs. So, the code that we jump to when we take an interrupt, as well as space for a stack to push context onto have to be available in both page tables. And finally, of course, we need to be able to figure out somehow what the other page table we should switch to is when we enter the kernel.
When we looked at the patches for Linux (and also the DragonFlyBSD patches at the time) on Friday and started asking questions, it became pretty evident that the initial work done by both was done under time constraints. Both had left the full kernel text mapped in both page tables, and the Linux trampoline design seemed over-complex. I started talking over some ideas with Robert Mustacchi about ways to fix these and who we should talk to, and reached out to some of my old workmates from the University of Queensland who were involved with OpenBSD. It seemed to me that the OpenBSD developers would care about these issues even more than we did, and would want to work out how to do the mitigation right.
I ended up sending an email to Philip Guenther on Friday afternoon, and on Saturday morning I drove an hour or so to meet up with him for coffee to talk page tables and interrupt trampolines. We wound up spending a good 6 hours at the coffee shop, and I came back with several pages of notes and a half-decent idea of the shape of the work to come.
One detail we missed that day was the interaction of per-CPU structures with per-process page tables. Much of the interrupt trampoline work is most easily done by using per-CPU structures in memory (and you definitely want a per-CPU stack!). If you combine that with per-process page tables, however, you have a problem: if you leave all the per-CPU areas mapped in all the processes, you will leak information (via Meltdown) about the state of one process to a different one when taking interrupts. In particular, you will leak things like %rip, which ruins all the work being done with PIE and ASLR pretty quickly. So, there are two options: you can either allocate the per-CPU structures per-process (so you end up with $NCPUS * $NPROCS of them); or you can make the page tables per-CPU.
OpenBSD, like Linux and the other implementations so far, decided to go down the road of per-CPU per-process pages to solve this issue. For illumos, we took the other route.
In illumos, it turned out that we already had per-CPU page tables. Robert and I re-discovered this on the Sunday of that week. We use them for 32-bit processes due to having full P>V PAE support in our kernel (which is, as it turns out, relatively uncommon amongst open-source OS). The logic to deal with creating and managing them and updating them was all already written, and after reading the code we concluded we could basically make a few small changes and re-use all of it. So we did.
By the end of that second week, we had a prototype that could get to userland. But, when working on this kind of kernel change we have a rule of thumb we use: after the first 70% of the patch is done and we can boot again, now it's time for the second 70%. In fact it turned out to be more like the second 200% for us -- a tedious long tail of bugs to solve that ended up necessitating some changes in the design as well.
At first we borrowed the method that Matt Dillon used for DragonFlyBSD, by putting the temporary "stack" space and state data for the interrupt trampolines into an extra page tacked onto the end of *%gs (in illumos the structure that lives there is the cpu_t).
If you read the existing logic in interrupt handlers for dealing with %gs though, you will quickly notice that the corner cases start to build up. There are a bunch of situations where the kernel temporarily alters %gs, and some of the ways to mess it up have security consequences that end up being worse than the bug we're trying to fix. As it turns out, there are no less than 3 different ways that ISRs use to try to get to having the right cpu_t in %gs on illumos, as it turns out, and they are all subtly different. Trying to tell which you should use when requires a bunch of test logic that in turn requires branches and changes to the CPU state, which is difficult to do in a trampoline where you're trying to avoid altering that state as much as possible until you've got the real stack online to push things into.
I kept in touch with Philip Guenther and Mike Larkin from the OpenBSD project throughout the weeks that followed. In one of the discussions we had, we talked about the NMI/MCE handlers and the fact that their handling currently on OpenBSD neglected some nasty corner-cases around interrupting an existing trap handler. A big part of the solution to those issues was to use a feature called IST, which allows you to unconditionally change stacks when you take an interrupt.
Traditionally, x86 only changes the stack pointer (%rsp on AMD64) while taking an interrupt when there is a privilege level change. If you take an interrupt while already in the kernel, the CPU does not change the stack pointer, and simply pushes the interrupt stack frame onto the stack you're already using. IST makes the change of stack pointer unconditional. If used unwisely, this is a bad idea: if you stay on that stack and turn interrupts back on, you could take another interrupt and clobber the frame you're already in. However, in it I saw a possible way to simplify the KPTI trampoline logic and avoid having to deal with %gs.
A few weeks into the project, John Levon joined us at work. He had previously worked on a bunch of Xen-related stuff as well as other parts of the kernel very close to where we were, so he quickly got up to speed with the KPTI work as well. He and I drafted out a "crazy idea" on the whiteboard one afternoon where we would use IST for all interrupts on the system, and put the "stack" they used in the KPTI page on the end of the cpu_t. Then, they could easily use stack-relative addresses to get the page table to change to, then pivot their stack to the real kernel stack memory, and throw away (almost) all the conditional logic. A few days later, we had convinced each other that this was the way to go.
Two of the most annoying x86 issues we had to work around were related to the SYSENTER instruction. This instruction is used to make "fast" system calls in 32-bit userland. It has a couple of unfortunate properties: firstly, it doesn't save or restore RFLAGS, so the kernel code has to take care of this (and be very careful not to clobber any of it before saving or after restoring it). Secondly, if you execute SYSENTER with the TF ("trap"/single-step flag) set by a debugger, the resulting debug trap's frame points at kernel code instead of the user code where it actually happened. The first one requires some careful gymnastics on the entry and return trampolines specifically for SYSENTER, while the second is a nasty case that is incidentally made easier by using IST. With IST, we can simply make the debug trap trampoline check for whether we took the trap in another trampoline's code, and reset %cr3 and the destination stack. This works for single-stepping into any of the handlers, not just the one for SYSENTER.
To make debugging easier, we decided that traps like the debug/single-step trap (as well as faults like page faults, #GP, etc.) would push their interrupt frame in a different part of the KPTI state page to normal interrupts. We applied this change to all the traps that can interrupt another trampoline (based on the instructions we used). These "paranoid" traps also set a flag in the KPTI struct to mark it busy (and jump to the double-fault handler if it is), to work around some bugs where double-faults are not correctly generated.
It's been a long and busy two months, with lots of time spent building, testing, and validating the code. We've run it on as many kinds of machines as we could get our hands on, to try to make sure we catch issues. The time we've spent on this has been validated several times in the process by finding bugs that could have been nasty in production.
One great example: our patches on Westmere-EP Xeons were causing busy machines to throw a lot of L0 I-cache parity errors. This seemed very mysterious at first, and it took us a few times seeing it to believe that it was actually our fault. This was actually caused by the accidental activation of a CPU errata for Westmere (B52, "Memory Aliasing of Code Pages May Cause Unpredictable System Behaviour") -- it turned out we had made a typo and put the "cacheable" flag into a variable named flags instead of attrs where it belonged when setting up the page tables. This was causing performance degradation on other machines, but on Westmere it causes cache parity errors as well. This is a great example of the surprising consequences that small mistakes in this kind of code can end up having. In the end, I'm glad that that erratum existed, otherwise it may have been a long time before we caught that bug.
As of this week, Mike and Philip have committed the OpenBSD patches for KPTI to their repository, and the patches for illumos are out for review. It's a nice kind of symmetry that the two projects who started on the work to
Wed, 21 Mar 2018 - 2h 03min - 837 - 237: AsiaBSDcon 2018
AsiaBSDcon review, Meltdown and Spectre Patches in FreeBSD stable, Interview with MidnightBSD founder, 8 months with TrueOS, mysteries of GNU and BSD split
This episode was brought to you by
Headlines
AsiaBSDCon 2018 has concluded
We have just returned from AsiaBSDCon in Tokyo, Japan last weekend Please excuse our jetlag The conference consisted two days of meeting followed by 2 days of paper presentations We arrived a few days early to see some sights and take a few extra delicious meals in Tokyo The first day of meetings was a FreeBSD developer summit (while Benedict was teaching his two tutorials) where we discussed the FreeBSD release cycle and our thoughts on improving it, the new Casper capsicum helper service, and developments in SDIO which will eventually enable WiFi and SD card readers on more embedded devices The second day of meetings consisted of bhyvecon, a miniconf that covered development in all hypervisors on all BSDs. It also included presentations on the porting of bhyve to IllumOS. Then the conference started There were a number of great presentations, plus an amazing hallway track as usual It was great to see many old friends and to spend time discussing the latest happenings in BSD. A couple of people came by and asked to take a picture with us and we were happy to do that. ***FreeBSD releases Spectre and Meltdown mitigations for 11.1
Speculative execution vulnerability mitigation is a work in progress. This advisory addresses the most significant issues for FreeBSD 11.1 on amd64 CPUs. We expect to update this advisory to include 10.x for amd64 CPUs. Future FreeBSD releases will address this issue on i386 and other CPUs. freebsd-update will include changes on i386 as part of this update due to common code changes shared between amd64 and i386, however it contains no functional changes for i386 (in particular, it does not mitigate the issue on i386).
Meltdown: The mitigation is known as Page Table Isolation (PTI). PTI largely separates kernel and user mode page tables, so that even during speculative execution most of the kernel's data is unmapped and not accessible. A demonstration of the Meltdown vulnerability is available at https://github.com/dag-erling/meltdown. A positive result is definitive (that is, the vulnerability exists with certainty). A negative result indicates either that the CPU is not affected, or that the test is not capable of demonstrating the issue on the CPU (and may need to be modified). A patched kernel will automatically enable PTI on Intel CPUs. The status can be checked via the vm.pmap.pti sysctl PTI introduces a performance regression. The observed performance loss is significant in microbenchmarks of system call overhead, but is much smaller for many real workloads. Spectre V2: There are two common mitigations for Spectre V2. This patch includes a mitigation using Indirect Branch Restricted Speculation, a feature available via a microcode update from processor manufacturers. The alternate mitigation, Retpoline, is a feature available in newer compilers. The feasibility of applying Retpoline to stable branches and/or releases is under investigation. The patch includes the IBRS mitigation for Spectre V2. To use the mitigation the system must have an updated microcode; with older microcode a patched kernel will function without the mitigation. IBRS can be disabled via the hw.ibrs_disable sysctl (and tunable), and the status can be checked via the hw.ibrs_active sysctl. IBRS may be enabled or disabled at runtime. Additional detail on microcode updates will follow. Wiki tracking the vulnerabilities and mitigations on different platforms ***
Many modern processors have implementation issues that allow unprivileged attackers to bypass user-kernel or inter-process memory access restrictions by exploiting speculative execution and shared resources (for example, caches).
An attacker may be able to read secret data from the kernel or from a process when executing untrusted code (for example, in a web browser).Interview with MidnightBSD Founder and Lead Dev Lucas Holt
[Authors Note: For those who havent heard about this, let me summarize. Last month, someone from the OpenBSD team added the Pale Moon browser to their ports collection. A Pale Moon developer demanded that they include Pale Moons libraries instead of using system libraries. As the conversation continued, it got more hostile, especially on the Pale Moon side. The net result is that Pale Moon will not be available on OpenBSD, MidnightBSD, or FreeBSD.]Recently, I have taken a little dip into the world of BSD. As part of my attempt to understand the BSD world a little better, I connected with Lucas Holt (MidnightBSD founder and lead developer) to ask him a few questions about his project. Here are his answers.
Its FOSS: Please explain MidnightBSD in a nutshell. How is it different than other BSDs?
Lucas Holt: MidnightBSD is a desktop focused operating system. When its considered stable, it will provide a full desktop experience. This differs from other efforts such as TrueOS or GhostBSD in that its not a distro of FreeBSD, but rather a fork. MidnightBSD has its own package manager, mport as well as unique package cluster software and several features built into user land such as mDNSresponder, libdispatch, and customizations throughout the system.
Its FOSS: Who is MidnightBSD aimed at?
Lucas Holt: The goal with MidnightBSD has always been to provide a desktop OS thats usable for everyday tasks and that even somewhat non technical people can use. Early versions of Mac OS X were certainly an inspiration. In practice, were rather far from that goal at this point, but its been an excellent learning opportunity.
Its FOSS: What is your background in computers?
Lucas Holt: I started in technical support at a small ISP and moved into web design and system administration. While there, I learned BSDi, Solaris and Linux. I also started tinkering with programming web apps in ASP and a little perl CGI. I then did a mix of programming and system administration jobs through college and graduated with a bachelors in C.S. from Eastern Michigan University. During that time, I learned NetBSD and FreeBSD. I started working on several projects such as porting Apples HFS+ code to FreeBSD 6 and working on getting the nforce2 chipset SATA controller working with FreeBSD 6, with the latter getting committed. I got a real taste for BSD and after seeing the lack of interest in the community for desktop BSDs, I started MidnightBSD. I began work on it in late 2005.
Currently, Im a Senior Software Engineer focusing on backend rest services by day and a part-time graduate student at the University of Michigan Flint.Its FOSS: I recently installed TrueOS. I was disappointed that a couple of the programs I wanted were not available. The FreeBSD port system looked mildly complicated for beginners. Im used to using pacman to get the job done quickly. How does MidnightBSD deal with ports?
Lucas Holt: MidnightBSD has its own port system, mports, which shared similarities with FreeBSD ports as well as some ideas from OpenBSD. We decided early on that decent package management was essential for regular users. Power users will still use ports for certain software, but its just so time consuming to build everything. We started work on our own package manager, mport.
Every package is a tar lzma archive with a sqlite3 manifest file as well as a sqlite 3 index thats downloaded from our server. This allows users to query and customize the package system with standard SQL queries. Were also building more user friendly graphical tools.
Package availability is another issue that most BSDs have. Software tends to be written for one or two operating systems and many projects are reluctant to support other systems, particularly smaller projects like MidnightBSD. There are certainly gaps. All of the BSD projects need more volunteers to help with porting software and keeping it up to date.Its FOSS: During your June 2015 interview on BSDNow, you mentioned that even though you support both i386 and amd64, that you recommend people choose amd64. Do you have any plans to drop i386 support in the future, like many have done?
Lucas Holt: Yes, we do plan to drop i386 support, mostly because of the extra work needed to build and maintain packages. Ive held off on this so far because I had a lot of feedback from users in South America that they still needed it. For now, the plan is to keep i386 support through 1.0 release. Thats probably a year or two out.
Its FOSS: What desktop environments does MidnightBSD support?
Lucas Holt: The original plan was to use Etoile as a desktop environment, but that project changed focus. We currently support Xfce, Gnome 3, WindowMaker + GNUstep + Gworkspace as primary choices. We also have several other window managers and desktop environments available such as Enlightenment, rat poison, afterstep, etc.
Early versions offered KDE 3.x but we had some issues with KDE 4. We may revisit that with newer versions.Its FOSS: What is MidnightBSDs default filesystem? Do you support DragonflyBSDs HAMMER filesystem? What other filesystems?
Lucas Holt: Boot volumes are UFS2. We also support ZFS for additional storage. We have read support for ExFat, NTFS, ext2, CD9660. NFS v3 and v4 are also supported for network file systems.
We do not support HAMMER, although it was considered. I would love to see HAMMER2 get added to MidnightBSD eventually.Its FOSS: Is MidnightBSD affected by the recent Spectre and Meltdown issues?
Lucas Holt: Yes. Most operating systems were affected by these issues. We were not informed of the issue until the general public became aware. Work is ongoing to come up with appropriate mitigations. Unfortunately, we do not have a patch yet.
Its FOSS: The Raspberry Pi and its many clones have made the ARM platform very popular. Are there any plans to make MidnightBSD available on that platform?
Lucas Holt: No immediate plans. ARM is an interesting architecture, but by the very nature of SoC designs, takes a lot of work to support a broad number of devices. It might be possible when we stop supporting i386 or if someone volunteers to work on the ARM port.
Eventually, I think most hobby systems will need to run ARM chips. Intels planning on locking down hardware with UEFI 3 and this may make it difficult to run on commodity hardware in the future not only for MidnightBSD but other systems as well.
At one point, MidinightBSD ran on sparc64. When workstations were killed off, we dropped support. A desktop OS on a server platform makes little sense.Its FOSS: Does MidnightBSD offer support for Linux applications?
Lucas Holt: Yes, we offer Linux emulation. Its emulating a 2.6.16 kernel currently and that needs to be updated so support newer apps. Its possible to run semi-recent versions of Firefox, Thunderbird, Java, and OpenOffice on it though. Ive also used it to host game servers in the past and play older games such as Quake 3, enemy territory, etc.
Its FOSS: Could you comment on the recent dust-up between the Pale Moon browser developers and the team behind the OpenBSD ports system?
Lucas Holt: I found this discussion frustrating. Many of the BSD projects hear a lot of complaints about browser availability and compatibility. With Firefox moving to Rust, it makes it even more difficult. Then you get into branding issues. Like Firefox, the Pale Moon developers have decided to protect their brand at the cost of users. Unlike the Firefox devs, theyve made even stranger requirements for branding. It is not possible to use a system library version of anything with Pale Moon and keep their branding requirements. As such, we cannot offer Pale Moon in MidnightBSD.
The reason this is an issue for an open source project is that many third party libraries are used in something as complex as a web browser. For instance, Gecko-based browsers use several multimedia libraries, sqlite3 (for bookmarks), audio and video codecs, etc. Trying to maintain upstream patches for each of these items is difficult. Thats why the BSDs have ports collections to begin with. It allows us to track and manage custom patches to make all these libraries work. We go through a lot of effort in keeping these up to date. Sometimes upstream patches dont get included. That means our versions are the only working copies. With pale moons policy, wed need to submit separate patches to their customized versions of all these libraries too and any new release of the browser would not be available as changes occur. It might not even be possible to compile pale moon without a patch locally.
With regard to Rust, it requires porting the language, as well as an appropriate version of LLVM before you can even start on the browser.Its FOSS: If someone wanted to contribute to your project, both financial and technical, how can they do that?
Lucas Holt: Financial assistance for the project can be submitted online. We have a page outlining how to make donations with Patreon, Paypal or via bitcoin. Donations are not tax deductible. You can learn more at http://www.midnightbsd.org/donate/
We also need assistance with translations, porting applications, and working on the actual OS. Interested parties can contact us on the mailing list or through IRC on freenode #midnightbsd We also could use assistance with mirroring ISOs and packages.I would like to thank Lucas for taking the time to reply to my many questions. For more information about MidnightBSD or to download it, please visit their website. The most recent version of MidnightBSD is 0.8.6.
News Roundup
8 months with TrueOS
Purpose of this review - what it is and what it is not.
The primary tasks I get done with my computer.I vowed to write down what I felt about TrueOS if I ever got to the six month mark of usage. This is just that. This is neither a tutorial, nor a piece of evangelism dedicated towards it.
This is also not a review of specific parts of TrueOS such as Lumina or AppCafe, since I don't use them at all.
In the spirit of presenting a screen shot, here is my i3wm displaying 4 windows in one screen - a configuration that I never use. https://inflo.ws/blog/images/trues-screenshot.png
I need a tiling wm with multi-desktop capability. As regards what I do with a computer, it is fairly straightforward to describe if I just list down my most frequently used applications.
xterm (CLI) Emacs (General editing and org mode) Intellij IDEA (Java, Kotlin, SQL) Firefox (Main web browser, with Multi-Account Containers) Thunderbird (Work e-mail) Notmuchmail (Personal e-mail) Chromium/Iridium (Dumb web browser) Telegram Desktop weechat (with wee-slack) cmus (Music player) mpv (Video player) mps-youtube (Youtube client) transmission-gtk Postgresql10 (daemon) Rabbitmq (daemon) Seafile (file sync) Shotwell (manage pictures) GIMP (Edit pictures) Calibre (Manage e-books) VirtualBox
Why did I pick TrueOS ?All of these are available as binary packages from the repository. Since I use Intellij Ultimate edition, I decided to download the no-jdk linux version from the website rather than install it. This would make sure that it gets updated regularly.
Must run Intellij IDEA. Must have proper graphics and sound driver support. Must be able to run VirtualBox.I ran various Linux distributions from 2001 all the way till 2009, till I discovered Arch, and continued with it till 2017. I tried out Void for two months before I switched to TrueOS.
Over the last few years, I started feeling like no matter which Linux distribution I touched, they all just stopped making a lot of sense. Generally in the way things were organised, and particularly in terms of software like systemd, which just got pushed down my throat. I couldn't wrap my head around half the things going on in my computer.
Mostly I found that Linux distributions stopped becoming a collection of applications that got developed together to something more coupled by software mechanisms like systemd - and that process was more and more opaque. I don't want to talk about the merits and de-merits of systemd, lets just say that I found it of no use and an unnecessary hassle.
In February, I found myself in charge of the entire technology stack of a company, and I was free to make choices. A friend who was a long time FreeBSD user convinced me to try it on the servers. My requirement then was to run Postgres, Rabbitmq, Nginx and a couple of JVM processes. The setup was zero hassle and it hasn't changed much in a year.
About three months of running FreeBSD-11.x on servers was enough for me to consider it for my laptop. I was very apprehensive of hardware support, but luckily my computer is a Thinkpad, and Thinkpads sort of work out of the box with various BSDs.
My general requirements were:
I had to pick from FreeBSD, NetBSD and OpenBSD, since these were the major BSDs that I was familiar with. One of my requirements was that I needed to be able to run VMs just in case I needed to test something on Windows/Linux. This ruled out OpenBSD. Then I was left with NetBSD and FreeBSD. NetBSD's driver support for newer Intel chip-sets were questionable, and FreeBSD was the only choice then.
When I was digging through FreeBSD forums, I found out that running the 11.x RELEASE on my laptop was out of the question since it didn't have proper drivers for my chip-set either.
A few more hours of digging led me to GhostBSD and TrueOS. I picked TrueOS straightaway because - well because TrueOS came from the old PC-BSD and it was built off FreeBSD-12-CURRENT with the latest drivers integrated.
I downloaded the UNSTABLE version available in June 2017, backed up ALL my data and home directory, and then installed it. There were no glitches during installation - I simply followed the installation as described in the handbook and everything was fine.
My entire switch from Arch/Void to TrueOS took about an hour, discounting the time it took to backup my data to an external hard disk. It was that easy. Everything I wanted to work just worked, everything was available in the repo.
Tweaks from cooltrainer.org : I discovered this excellent tutorial that describes setting up a FreeBSD 11 desktop. It documents several useful tweaks, some of which I applied. A few examples - Fonts, VirtualBox, Firewall, UTF-8 sections.
TrueOS (and FreeBSD) specific things I liked
Open-rc
Boot EnvironmentsThe open-rc init system is familiar and is well documented.
TrueOS specific parts are described here.
When I installed postgresql10-server, there was no open-rc script for it, but I could cobble one together in two hours with zero prior experience writing init scripts. Later on I figured out that the init script for postgresql9 would work for 10 as well, and used that.
ZFSThis was an alien concept to me, but the first time I did an update without waiting for a CDN sync to finish, my computer booted into the shell and remained there. The friendly people at TrueOS discourse asked me to roll back to an older BE and wait for sync to finish.
I dug through the forums and found "ZFS / Snapshots basics & How-Tos for those new to TrueOS". This describes ZFS and BEs, and is well worth reading.
File-system layoutMy experience with boot environments was enough to convince me about the utility of ZFS. I am still reading about it and trying things out, and whatever I read just convinces me more about why it is good.
Coming from the Linux world, how the FreeBSD file-system is laid out seemed odd at first. Then I realised that it was the Linux distros that were doing the odd thing. e.g : The whole OS is split into base system and applications. All the non base system configurations and apps go into /usr/local. That made a lot of sense.
The entire OS is developed along with its applications asWed, 14 Mar 2018 - 1h 39min - 836 - 236: How a cd works
Well cover OpenBSDs defensive approach to OS security, help you Understanding Syscall Conventions for Different Platforms, Mishandling SMTP Sender Verification, how the cd command works, and the LUA boot loader coming to FreeBSD.
This episode was brought to you by
Headlines
Pledge: OpenBSDs defensive approach to OS Security
How does it make a program more secure?The meaning of Pledge is same as in the real world, that is, a solemn promise or undertaking.
So, in OpenBSD: Calling pledge in a program means to promise that the program will only use certain resources.
You added pledge to use only stdio and nothing else. Then, a malicious user found out that there is a vulnerability in your program which one can exploit and get into shell (or root shell). Exploiting your program to open a shell (or root shell) will result in the kernel killing the process with SIGABRT (which cannot be caught/ignored) and will generate a log (which you can find with dmesg).It limits the operation of a program. Example: You wrote a program named abc that only needed the stdio to just print something to stdout.
pledge(read,NULL) ? wrong syntax of the pledge() pledge(stdio inet,NULL) ? correct syntax of the pledge()This happens because before executing other codes of your program, the code first pledges not to use anything other than stdio promise/operations. But, opening a shell or root shell will call several other system-calls which are distributed in lots of other promises like stdio, proc, exec etc. They are all forbidden because the program has already promised not to use any promises other than stdio.
Pledge is not a system call filter. So, it is not used to restrict system calls.
For example,Pledge works on stdio, dns, inet, etc. promises but not directly on system calls like read, write, etc. And, unique functionality of pledge() is that it works on behavioral approach not just like 1:1 approach with the system calls.
On 11 December 2017, Theo de Raadt said:
List: openbsd-tech Subject: pledge execpromises From: Theo de Raadt <deraadt () openbsd ! org> Date: 20171211 21:20:51 Message-ID: 6735.1513027251 () cvs ! openbsd ! org This will probably be committed in the next day or so. The 2nd argument of pledge() becomes execpromises, which is what will gets activated after execve. There is also a small new feature called error, which causes violating system calls to return -1 with ENOSYS rather than killing the process. This must be used with EXTREME CAUTION because libraries and programs are full of unchecked system calls. If you carry on past one of these failures, your program is in uncharted territory and risks of exploitation become high. error is being introduced for a different reason: The pre-exec processs expectation of what the post-exec process will do might mismatch, so error allows things like starting an editor which has no network access or maybe other restrictions in the futureEvery Journey Starts with a FAIL...or Understanding Syscall Conventions for Different Platforms
Introduction
The ProblemNot long ago I started looking into FreeBSD kernel exploitation. There are only a few resources but probably the best starting point is argp's Phrack article from 2009[0]. And while he does only provide one technique, I wanted to understand it and port it to a modern FreeBSD release before describing new, own researched techniques.
Well, at least this was my plan. In reality I ended researching how different operating systems resp. the same operating system but for different architectures implement syscalls. Hence, new exploiting methods have to wait for another post. In this one I want to describe my personal FAIL while porting argp's exploit example to a FreeBSD 11.1-RELEASE running on a 64bit processor. Maybe this will give other people interested in kernel stuff some insights they didn't know before. If you already know how syscalls work on 32bit and 64bit *BSD because you are an experienced exploit or kernel developer, you will probably want to search for something
else to read. Moreover, some of the debugging stuff can look laborious because I wanted to show the steps I have done while attacking my problem instead of showing a simple walkthrough to the solution.
Why the Original Code Worked While It Was Wrongargp described in his article vulnerable code consisting of a loadable kernel module which exposes a syscall to the userland. Because it was written around the time when FreeBSD 8-RELEASE came out and because he has written himself that the code needs smaller adjustments to work with this version (it was written for FreeBSD 7) I thought I will first port it to
FreeBSD 11.1-RELEASE. Moreover it was written for an Intel 32bit processor architecture as we can see from his shellcode examples. Hence, I wanted to go right away the harder way and modify it to work on an 64bit processor.As written above, the syscall convention for the 32bit architecture is different from the one for the 64bit architecture. Indeed, a syscall on a 32bit FreeBSD system passes the arguments via the stack while the syscall offset is stored in the EAX register. The transfer into the kernel address space is done in 'cpu_fetch_syscall_args' in 'sys/i386/i386/trap.c'.
int cpu_fetch_syscall_args(struct thread *td, struct syscall_args *sa) { ... frame = td->td_frame; params = (caddr_t)frame->tf_esp + sizeof(int); sa->code = frame->tf_eax; ... if (params != NULL && sa->narg != 0) error = copyin(params, (caddr_t)sa->args, (u_int)(sa->narg * sizeof(int))); else ... }That is, 'params' points to ESP+4 bytes offset. Later, the arguments are copied into the kernel space which is referenced by 'sa->args'. 'args' is an array of eight 'register_t' which is defined as 'int32_t' on the 32bit platform in comparison to the 64bit platform. And as 'struct args' only
consisted of integers they got copied into the syscall arguments which are given to the trigger function inside the kernel module. We could verify this by changing 'int op' to 'long long op' in the kernel module and in trigger.c. We get the following output:root@freebsd64:trigger/ # ./trigger 0x28414000 256 3 1 0x28414000 256 4294967295 2 root@freebsd64:trigger/ #To bring this to an end: argp's version only worked for his special choice of arguments and only on 32bit. On 32bit FreeBSD platforms the arguments are transferred into kernel space by 4 byte integers, hence it will only work for integers anyway. On 64bit FreeBSD platforms we have to use syscall(2) in the intended way.
iXsystems
New Disks!A Life Lesson in Mishandling SMTP Sender Verification
It all started with one of those rare spam mails that got through.
This one was hawking address lists, much like the ones I occasionally receive to addresses that I can not turn into spamtraps. The message was addressed to, of all things, root@skapet.bsdly.net. (The message with full headers has been preserved here for reference).
Yes, that's right, they sent their spam to root@. And a quick peek at the headers revealed that like most of those attempts at hawking address lists for spamming that actually make it to a mailbox here, this one had been sent by an outlook.com customer.
The problem with spam delivered via outlook.com is that you can't usefully blacklist the sending server, since the largish chunk of the world that uses some sort of Microsoft hosted email solution (Office365 and its ilk) have their usually legitimate mail delivered via the very same infrastructure.
And since outlook.com is one of the mail providers that doesn't play well with greylisting (it spreads its retries across no less than 81 subnets (the output of 'echo outlook.com | doas smtpctl spf walk' is preserved here), it's fairly common practice to just whitelist all those networks and avoid the hassle of lost or delayed mail to and from Microsoft customers.
I was going to just ignore this message too, but we've seen an increasing number of spammy outfits taking advantage of outlook.com's seeming right of way to innocent third parties' mail boxes.
So I decided to try both to do my best at demoralizing this particular sender and alert outlook.com to their problem. I wrote a messsage (preserved here) with a Cc: to abuse@outlook.com where the meat is,
Ms Farell, The address root@skapet.bsdly.net has never been subscribed to any mailing list, for obvious reasons. Whoever sold you an address list with that address on it are criminals and you should at least demand your money back. Whoever handles abuse@outlook.com will appreciate the attachment, which is a copy of the message as it arrived here with all headers intact. Yours sincerely, Peter N. M. HansteenWhat happened next is quite amazing.
If my analysis is correct, it may not be possible for senders who are not themselves outlook.com customers to actually reach the outlook.com abuse team.
Any student or practitioner of SMTP mail delivery should know that SPF records should only happen on ingress, that is at the point where the mail traffic enters your infrastructure and the sender IP address is the original one. Leave the check for later when the message may have been forwarded, and you do not have sufficient data to perform the check.
Whenever I encounter incredibly stupid and functionally destructive configuration errors like this I tend to believe they're down to simple incompetence and not malice.
But this one has me wondering. If you essentially require incoming mail to include the contents of spf.outlook.com (currently no less than 81 subnets) as valid senders for the domain, you are essentially saying that only outlook.com customers are allowed to communicate.
If that restriction is a result of a deliberate choice rather than a simple configuration error, the problem moves out of the technical sphere and could conceivably become a legal matter, depending on what outlook.com have specified in their contracts that they are selling to their customers.
But let us assume that this is indeed a matter of simple bad luck or incompetence and that the solution is indeed technical.
I would have liked to report this to whoever does technical things at that domain via email, but unfortunately there are indications that being their customer is a precondition for using that channel of communication to them.
I hope they fix that, and soon. And then move on to terminating their spamming customers' contracts.
The main lesson to be learned from this is that when you shop around for email service, please do yourself a favor and make an effort to ensure that your prospective providers actually understand how the modern-ish SMTP addons SPF, DKIM and DMARC actually work.
Otherwise you may end up receiving more of the mail you don't want than what you do want, and your own mail may end up not being delivered as intended.
News Roundup
Running Salt Proxy Minions on OpenBSD
Installation and StartupAs I have previously attempted several times in the past, I am (finally) very close to switch to OpenBSD, a more stable and reliable operating system that I like. Before starting to make the actual change on both personal and work computer, I started testing some of the tools Im currently using, and understand what are the expectations.
In general I didnt encounter issues, or when I did, I found the answers in the documentation (which is really great), or various forums. I didnt find however any questions regarding Proxy Minions on OpenBSD which is why I thought it might be helpful to share my experience.
Starting up the Proxy MinionsWith these said, I started playing with Salt, and it was simple and straightforward. First step - install Salt: pkg_add salt. This will bring several ports for Python futures, ZeroMQ, or Tornado which are needed for Salt.
After configuring the pillar_roots in the /etc/salt/master config file for the Master, I started up the master process using rcctl:
Starting many Proxy MinionsThe Salt package for OpenBSD comes with the rc file for salt-proxy as well, /etc/rc.d/salt_proxy
While typically you run a single regular Minion on a given machine, it is very like that there are multiple Proxy processes. Additionally, the default Salt rc file has the following configuration for the salt-proxy daemon:
Have a separate rc file per Proxy, each having the daemon instruction explicitly specifying its Minion ID Start the service (using the regular Minion that controls the machine where the Proxy processes are running) And the test Proxy Minion is then up (after accepting the key, i.e,, salt-key -a test)I have managed to startup a Proxy Minion, but what about many? Executing the three commands above for each and every device is tedious and cannot scale very well. I thus have figured the following way:
Using the file.managed State function to generate the contents of the rc file for each Proxy, with its own Minion ID. Using the service.running State function start the service.Extending the same to a (very) large number of Proxy Minions, you can easily manage the rc files and start the services using a Salt State executed on the regular Minion:
ConclusionsThese two steps would suffice to start an arbitrary number of Proxy Minions, and the command executed will always be the same regardless how many processes you aim to manage.
I am still a novice when it comes to OpenBSD, I have plenty to learn, but it looks like the transition will be much smoother than I expected. I am already looking forward to the handover, and - most importantly - I will no longer be using systemd. :-)
LUA boot loader coming very soon
As you may know, the Lua (http://www.lua.org) boot loader has been in the works for some time. It started out life as a GSoC in 2014 by Pedro Souza mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to try to integrate it. I rebased that effort into a github branch which Pedro Arthur fixed up. Over the past year, I've been cleaning up the boot loader
for other reasons, and found the time was ripe to start integrating this into the tree. However, those integration efforts have taken a while as my day-job work on the boot loader took priority. In the mean time, Ed Maste and the FreeBSD Foundation funded Zakary Nafziger to enhance the original GSoC Lua scripts to bring it closer to parity with the evolution of the FORTH menu system since the GSoC project started.I'm pleased to announce that all these threads of development have converged and I'll be pushing the FreeBSD Lua Loader later today. This loader uses Lua as its scripting language instead of FORTH. While co-existance is planned, the timeline for it is looking to be a few weeks
and I didn't want to delay pushing this into the tree for that.To try the loader, you'll need to build WITHOUT_FORTH=yes and WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this, you can do it in src/stand and install the result (be sure to have the options for both the build and the install). This will replace your current
/boot/loader that is scripted with FORTH to one that's scripted with Lua.
It will install the lua scripts in /boot/lua. The boot is scripted with /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised to create a backup copy of /boot/loader before testing (eg cp /boot/loader /boot/loader_forth), since you'll need to boot that from boot2 if something
goes wrong. I've tested it extensively, though, with userboot.so and it's test program, so all the initial kinks of finding the lua scripts, etc have been worked out.While it's possible to build all the /boot/loader variants with Lua, I've just tested a BIOS booting /boot/loader both with and without menus enabled. I've not tested any of the other variants and the instructions for testing some of them may be rather tedious (especially UEFI, if you want a
simple path to back out). Since there's not been full convergence testing, you'll almost certainly find bumps in this system. Also, all the build-system APIs are likely not yet final.I put MFC after a month on the commit. Due to the heroic (dare I say almost crazy) work of Kyle Evans on merging all the revs from -current to 11, I'm planning a MFC to 11 after the co-existence issues are hammered out. In 11, FORTH will be the default, and Lua will be built by default, but users will have to do something to use it. 12, both FORTH and Lua will be built and installed, with Lua as default (barring unforeseen complications). Once the co-existence stuff goes in, I imagine we'll make the switch to Lua by default shortly after that. In 13, FORTH will be removed unless there's a really really compelling case made to keep it.
So please give it a spin and give me any feedback, documentation updates and/or bug fixes. I'm especially interested in reviews from people that have embedded Lua in other projects or experts in Lua that can improve the robustness of the menu code.
Bitcoin Full Node on FreeBSD
What is a Bitcoin ?
What is a Bitcoin Wallet ?Bitcoin is a valuable popular open-source cryptocurrency that was invented by Satoshi Nakamoto in 2009. Bitcoins have value because they possess same characteristics like money (durability, portability, fungibility, scarcity, divisibility, and recognizability), but based on the properties of mathematics rather than on physical properties (like gold and silver) or trust in central authorities (like fiat currencies). In short, Bitcoin is backed by mathematics.
Bitcoin is the first decentralized peer-to-peer cryptocurrency that is controlled by its users.
Transactions take place directly between users, and are later verified by network nodes with digital signature and then placed in a public distributed ledger called a blockchain. Bitcoin is unique in that only 21 million bitcoins will ever be created. The unit of the bitcoin system is bitcoin or mBTC.A wallet is nothing more than a pair of public and private keys that are created by a client to store the digital credentials for your bitcoin.
There are several types of wallets:
What is a Blockchain?Desktop Wallet Token Wallet Online Wallet Mobile Wallet A token wallet is the safest way to work with bitcoin network, but you can use your mobile or pc as a bitcoin wallet.
What is Mining?A blockchain is a ledger that records bitcoin transactions. The blockchain is a distributed database that achieves independent verification of the chain of ownership. Each network node stores its own copy of the blockchain. Transactions will broadcast on the bitcoin network, and about 2400 transactions create a block. These blocks are building blocks of the blockchain.
Mining is the process of dedicating computing power to process transactions, secure the network, and keep everyone in the system synchronized together. It has been designed to be fully decentralized.
Miners need mining software with specialized hardware. Mining software listens for transactions broadcasted through the peer-to-peer network and performs appropriate tasks to process and confirm these transactions. Bitcoin miners perform this work because they can earn transaction fees paid by users for faster transaction processing.
New transactions have to be confirmed then be included in a block along with a mathematical proof of work. Such proofs are very hard to generate because there is no way to create them other than by trying billions of calculations per second. Hence, miners are required to perform these calculations before their blocks are accepted by the network and before they are rewarded. As more people start to mine, the difficulty of finding valid blocks is automatically increased by the network to ensure that the average time to find a block remains equal to 10 minutes. As a result, mining is a very competitive business where nWed, 07 Mar 2018 - 2h 01min - 835 - 235: I C you BSD
How the term open source was created, running FreeBSD on ThinkPad T530, Moving away from Windows, Unknown Giants, as well as OpenBSD and FreeDOS.
This episode was brought to you by
Headlines
How I coined the term 'open source'
Meetings on computer securityIn a few days, on February 3, the 20th anniversary of the introduction of the term "open source software" is upon us. As open source software grows in popularity and powers some of the most robust and important innovations of our time, we reflect on its rise to prominence.
I am the originator of the term "open source software" and came up with it while executive director at Foresight Institute. Not a software developer like the rest, I thank Linux programmer Todd Anderson for supporting the term and proposing it to the group.
This is my account of how I came up with it, how it was proposed, and the subsequent reactions. Of course, there are a number of accounts of the coining of the term, for example by Eric Raymond and Richard Stallman, yet this is mine, written on January 2, 2006.
It has never been published, until today.
The introduction of the term "open source software" was a deliberate effort to make this field of endeavor more understandable to newcomers and to business, which was viewed as necessary to its spread to a broader community of users. The problem with the main earlier label, "free software," was not its political connotations, but that—to newcomers—its seeming focus on price is distracting. A term was needed that focuses on the key issue of source code and that does not immediately confuse those new to the concept. The first term that came along at the right time and fulfilled these requirements was rapidly adopted: open source.
This term had long been used in an "intelligence" (i.e., spying) context, but to my knowledge, use of the term with respect to software prior to 1998 has not been confirmed. The account below describes how the term open source software caught on and became the name of both an industry and a movement.
Releasing NetscapeIn late 1997, weekly meetings were being held at Foresight Institute to discuss computer security. Foresight is a nonprofit think tank focused on nanotechnology and artificial intelligence, and software security is regarded as central to the reliability and security of both. We had identified free software as a promising approach to improving software security and reliability and were looking for ways to promote it. Interest in free software was starting to grow outside the programming community, and it was increasingly clear that an opportunity was coming to change the world. However, just how to do this was unclear, and we were groping for strategies.
At these meetings, we discussed the need for a new term due to the confusion factor. The argument was as follows: those new to the term "free software" assume it is referring to the price. Oldtimers must then launch into an explanation, usually given as follows: "We mean free as in freedom, not free as in beer." At this point, a discussion on software has turned into one about the price of an alcoholic beverage. The problem was not that explaining the meaning is impossible—the problem was that the name for an important idea should not be so confusing to newcomers. A clearer term was needed. No political issues were raised regarding the free software term; the issue was its lack of clarity to those new to the concept.
The key meetingOn February 2, 1998, Eric Raymond arrived on a visit to work with Netscape on the plan to release the browser code under a free-software-style license. We held a meeting that night at Foresight's office in Los Altos to strategize and refine our message. In addition to Eric and me, active participants included Brian Behlendorf, Michael Tiemann, Todd Anderson, Mark S. Miller, and Ka-Ping Yee. But at that meeting, the field was still described as free software or, by Brian, "source code available" software.
While in town, Eric used Foresight as a base of operations. At one point during his visit, he was called to the phone to talk with a couple of Netscape legal and/or marketing staff. When he was finished, I asked to be put on the phone with them—one man and one woman, perhaps Mitchell Baker—so I could bring up the need for a new term. They agreed in principle immediately, but no specific term was agreed upon.
Between meetings that week, I was still focused on the need for a better name and came up with the term "open source software." While not ideal, it struck me as good enough. I ran it by at least four others: Eric Drexler, Mark Miller, and Todd Anderson liked it, while a friend in marketing and public relations felt the term "open" had been overused and abused and believed we could do better. He was right in theory; however, I didn't have a better idea, so I thought I would try to go ahead and introduce it. In hindsight, I should have simply proposed it to Eric Raymond, but I didn't know him well at the time, so I took an indirect strategy instead.
Todd had agreed strongly about the need for a new term and offered to assist in getting the term introduced. This was helpful because, as a non-programmer, my influence within the free software community was weak. My work in nanotechnology education at Foresight was a plus, but not enough for me to be taken very seriously on free software questions. As a Linux programmer, Todd would be listened to more closely.
Getting the name outLater that week, on February 5, 1998, a group was assembled at VA Research to brainstorm on strategy. Attending—in addition to Eric Raymond, Todd, and me—were Larry Augustin, Sam Ockman, and attending by phone, Jon "maddog" Hall.
The primary topic was promotion strategy, especially which companies to approach. I said little, but was looking for an opportunity to introduce the proposed term. I felt that it wouldn't work for me to just blurt out, "All you technical people should start using my new term." Most of those attending didn't know me, and for all I knew, they might not even agree that a new term was greatly needed, or even somewhat desirable.
Fortunately, Todd was on the ball. Instead of making an assertion that the community should use this specific new term, he did something less directive—a smart thing to do with this community of strong-willed individuals. He simply used the term in a sentence on another topic—just dropped it into the conversation to see what happened. I went on alert, hoping for a response, but there was none at first. The discussion continued on the original topic. It seemed only he and I had noticed the usage.
Not so—memetic evolution was in action. A few minutes later, one of the others used the term, evidently without noticing, still discussing a topic other than terminology. Todd and I looked at each other out of the corners of our eyes to check: yes, we had both noticed what happened. I was excited—it might work! But I kept quiet: I still had low status in this group. Probably some were wondering why Eric had invited me at all.
Toward the end of the meeting, the question of terminology was brought up explicitly, probably by Todd or Eric. Maddog mentioned "freely distributable" as an earlier term, and "cooperatively developed" as a newer term. Eric listed "free software," "open source," and "sourceware" as the main options. Todd advocated the "open source" model, and Eric endorsed this. I didn't say much, letting Todd and Eric pull the (loose, informal) consensus together around the open source name. It was clear that to most of those at the meeting, the name change was not the most important thing discussed there; a relatively minor issue. Only about 10% of my notes from this meeting are on the terminology question.
But I was elated. These were some key leaders in the community, and they liked the new name, or at least didn't object. This was a very good sign. There was probably not much more I could do to help; Eric Raymond was far better positioned to spread the new meme, and he did. Bruce Perens signed on to the effort immediately, helping set up Opensource.org and playing a key role in spreading the new term.
For the name to succeed, it was necessary, or at least highly desirable, that Tim O'Reilly agree and actively use it in his many projects on behalf of the community. Also helpful would be use of the term in the upcoming official release of the Netscape Navigator code. By late February, both O'Reilly & Associates and Netscape had started to use the term.
A happy twingeAfter this, there was a period during which the term was promoted by Eric Raymond to the media, by Tim O'Reilly to business, and by both to the programming community. It seemed to spread very quickly.
On April 7, 1998, Tim O'Reilly held a meeting of key leaders in the field. Announced in advance as the first "Freeware Summit," by April 14 it was referred to as the first "Open Source Summit."
These months were extremely exciting for open source. Every week, it seemed, a new company announced plans to participate. Reading Slashdot became a necessity, even for those like me who were only peripherally involved. I strongly believe that the new term was helpful in enabling this rapid spread into business, which then enabled wider use by the public.
A quick Google search indicates that "open source" appears more often than "free software," but there still is substantial use of the free software term, which remains useful and should be included when communicating with audiences who prefer it.
When an early account of the terminology change written by Eric Raymond was posted on the Open Source Initiative website, I was listed as being at the VA brainstorming meeting, but not as the originator of the term. This was my own fault; I had neglected to tell Eric the details. My impulse was to let it pass and stay in the background, but Todd felt otherwise. He suggested to me that one day I would be glad to be known as the person who coined the name "open source software." He explained the situation to Eric, who promptly updated his site.
Coming up with a phrase is a small contribution, but I admit to being grateful to those who remember to credit me with it. Every time I hear it, which is very often now, it gives me a little happy twinge.
The big credit for persuading the community goes to Eric Raymond and Tim O'Reilly, who made it happen. Thanks to them for crediting me, and to Todd Anderson for his role throughout. The above is not a complete account of open source history; apologies to the many key players whose names do not appear. Those seeking a more complete account should refer to the links in this article and elsewhere on the net.
FreeBSD on a Laptop - A guide to a fully functional installation of FreeBSD on a ThinkPad T530
As I stated my previous post, I recently dug up my old ThinkPad T530 after the embarrassing stream of OS X security bugs this month. Although this ThinkPad ran Gentoo faithfully during my time in graduate school at Clemson, these days I'd much rather spend time my wife and baby than fighting with emerge and USE flags. FreeBSD has always been my OS of choice, and laptop support seems to be much better than it was a few years ago. In this guide, I'll show you the tweaks I made to wrestle FreeBSD into a decent experience on a laptop.
Unlike my usual posts, this time I'm going to assume you're already pretty familiar with FreeBSD. If you're a layman looking for your first BSD-based desktop, I highly recommend checking out TrueOS (previously PC-BSD): they've basically taken FreeBSD and packaged it with all the latest drivers, along with a user-friendly installer and custom desktop environment out of the box. TrueOS is an awesome project–the only reason I don't use it is because I'm old, grumpy, and persnickety about having my operating system just so.
Anyway, if you'd still like to take the plunge, read on. Keep in mind, I'm using a ThinkPad T530, but other ThinkPads of the same generation should be similarly compatible.
Here's what you'll get:
Decent battery life (8-9 hours with a new 9-cell battery) UEFI boot and full-disk encryption WiFi (Intel Ultimate-N 6300) Ethernet (Intel PRO/1000) Screen brightness adjustment Suspend/Resume on lid close (make sure to disable TPM in BIOS) Audio (Realtek ALC269 HDA, speakers and headphone jack) Keyboard multimedia buttons Touchpad/Trackpoint Graphics Acceleration (with integrated Intel graphics, NVIDIA card disabled in BIOS)What I haven't tested yet:
Bluetooth Webcam Fingerprint reader SD Card slotInstallation
Power Saving
Tweaks for Desktop Use
X11
Fonts
Login Manager: SLiM
Desktop Environment: i3
Applications
The LLVM Sanitizers stage accomplished
A prebuilt LLVM toolchainI've managed to get the Memory Sanitizer to work for the elementary base system utilities, like ps(1), awk(1) and ksh(1). This means that the toolchain is ready for tests and improvements. I've iterated over the basesystem utilities and I looked for bugs, both in programs and in sanitizers. The number of detected bugs in the userland programs was low, there merely was one reading of an uninitialized variable in ps(1).
Sanitizers Notable changes in sanitizers, all of them are in the context of NetBSD support.I've prepared a prebuilt toolchain with Clang, LLVM, LLDB and compiler-rt for NetBSD/amd64. I prepared the toolchain on 8.99.12, however I have received reports that it works on other older releases. Link: llvm-clang-compilerrt-lldb-7.0.0beta_2018-01-24.tar.bz2
The archive has to be untarballed to /usr/local (however it might work to some extent in other paths).
This toolchain contains a prebuilt tree of the LLVM projects from a snapshot of 7.0.0(svn). It is a pristine snapshot of HEAD with patches from pkgsrc-wip for llvm, clang, compiler-rt and lldb.
Base system changesAdded fstat(2) MSan interceptor. Support for kvm(3) interceptors in the common sanitizer code. Added devname(3) and devname_r(3) interceptors to the common sanitizer code. Added sysctl(3) familty of functions interceptors in the common sanitizer code. Added strlcpy(3)/strlcat(3) interceptors in the common sanitizer code. Added getgrouplist(3)/getgroupmembership(3) interceptors in the common sanitizer code. Correct ctype(3) interceptors in a code using Native Language Support. Correct tzset(3) interceptor in MSan. Correct localtime(3) interceptor in the common sanitizer code. Added paccept(2) interceptor to the common sanitizer code. Added access(2) and faccessat(2) interceptors to the common sanitizer code. Added acct(2) interceptor to the common sanitizer code. Added accept4(2) interceptor to the common sanitizer code. Added fgetln(3) interceptor to the common sanitizer code. Added interceptors for the pwcache(3)-style functions in the common sanitizer code. Added interceptors for the getprotoent(3)-style functions in the common sanitizer code. Added interceptors for the getnetent(3)-style functions in the common sanitizer code. Added interceptors for the fts(3)-style functions in the common sanitizer code. Added lstat(3) interceptor in MSan. Added strftime(3) interceptor in the common sanitizer code. Added strmode(3) interceptor in the common sanitizer code. Added interceptors for the regex(3)-style functions in the common sanitizer code. Disabled unwanted interceptor __sigsetjmp in TSan.I've tidied up inclusion of the internal namespace.h header in libc. This has hidden the usage of public global symbol names of:
strlcat -> _strlcat sysconf -> __sysconf closedir -> _closedir fparseln -> _fparseln kill -> _kill mkstemp -> _mkstemp reallocarr -> _reallocarr strcasecmp -> _strcasecmp strncasecmp -> _strncasecmp strptime -> _strptime strtok_r -> _strtok_r sysctl -> _sysctl dlopen -> __dlopen dlclose -> __dlclose dlsym -> __dlsym strlcpy -> _strlcpy fdopen -> _fdopen mmap -> _mmap strdup -> _strdup
Future directions and goals Possible paths in random order: In the quartet of UBSan (Undefined Behavior Sanitizer), ASan (Address Sanitizer), TSan (Thread Sanitizer), MSan (Memory Sanitizer) we need to add the fifth basic sanitizer: LSan (Leak Sanitizer). The Leak Sanitizer (detector of memory leaks) demands a stable ptrace(2) interface for processes with multiple threads (unless we want to build a custom kernel interface). Integrate the sanitizers with the userland framework in order to ship with the native toolchain to users. Port sanitizers from LLVM to GCC. Allow to sanitize programs linked against userland libraries other than libc, librt, libm and libpthread; by a global option (like MKSANITIZER) producing a userland that is partially prebuilt with a desired sanitizer. This is required to run e.g. MSanitized programs against editline(3). So far, there is no Operating System distribution in existence with a native integration with sanitizers. There are 3rd party scripts for certain OSes to build a stack of software dependencies in order to validate a piece of software. Execute ATF tests with the userland rebuilt with supported flavors of sanitizers and catch regressions. Finish porting of modern linkers designed for large C++ software, such as GNU GOLD and LLVM LLD. Today the bottleneck with building the LLVM toolchain is a suboptimal linker GNU ld(1).The purpose of these changes was to stop triggering interceptors recursively. Such interceptors lead to sanitization of internals of unprepared (not recompiled with sanitizers) prebuilt code. It's not trivial to sanitize libc's internals and the sanitizers are not designed to do so. This means that they are not a full replacement of Valgrind-like software, but a a supplement in the developer toolbox. Valgrind translates native code to a bytecode virtual machine, while sanitizers are designed to work with interceptors inside the pristine elementary libraries (libc, libm, librt, libpthread) and embed functionality into the executable's code.
I've also reverted the vadvise(2) syscall removal, from the previous month. This caused a regression in legacy code recompiled against still supported compat layers. Newly compiled code will use a libc's stub of vadvise(2).
I've also prepared a patch installing dedicated headers for sanitizers along with the base system GCC. It's still discussed and should land the sources soon.
Plan for the next milestone Keep upstreaming a pile of local compiler-rt patches. Restore the LLDB support for traced programs with a single thread.I've decided to not open new battlefields and return now to porting LLDB and fixing ptrace(2).
Interview - Goran Mekic - meka@tilda.center / @meka_floss
CBSD website Jail and VM Manager ***News Roundup
Finally Moving Away From Windows
Broken WindowThanks to a combination of some really impressive malware, bad clicking, and poor website choices, I had to blow away my Windows 10 installation. Not that it was Window’s fault, but a piece of malware had infected my computer when I tried to download a long lost driver for an even longer lost RAID card for a server. A word of advice – the download you’re looking for is never on an ad-infested forum in another language. In any case, I had been meaning to switch away from Windows soon. I didn’t have my entire plan ready, but now was as good a time as any.
My line of work requires me to maintain some form of Windows installation, so I decided to keep it in a VM rather than dual booting as I was developing code and not running any high-end visual stuff like games. My first thought was to install Arch or Gentoo Linux, but the last time I attempted a Gentoo installation it left me bootless. Not that there is anything wrong with Gentoo, it was probably my fault, but I like the idea of some sort of installer so I looked at rock-solid Debian. My dad had installed Debian on his sweet new cutting-edge Lenovo laptop he received recently from work. He often raves about his cool
Wed, 28 Feb 2018 - 2h 05min - 834 - 234: Code and Community
GSoC 2018 Projects announced, tutorial FreeBSD jails with iocage, new Code of Conduct for FreeBSD, libhijack, and fancy monitoring for OpenSMTPD
This episode was brought to you by
Headlines
Google Summer of Code 2018
FreeBSD FreeBSD Google Summer oF Code Ideas
NetBSDYou can join #freebsd-soc on the efnet IRC network to chat with FreeBSD developers interested in mentoring student proposals and projects, past FreeBSD/GSoC students, and other students applying to FreeBSD/GSoC this year.
You can get a stipend (paid for by Google) and spend a few months getting to know and improving the insides of NetBSD or pkgsrc.
The schedule is: 12-27 March Applying 23 April Find out if you were accepted 14 May - 22 August Do the project! We have some suggestions for suitable projects: - ARM EFI bootloader - Using libFuzzer on base tools - Refactoring ALTQ (QoS implementation) and integrating with NPF - Testsuite for libcurses - Improve pkgin Other suggestions and details are at: https://wiki.netbsd.org/projects/gsoc/
Haiku Students: How to Apply for a Haiku Idea Project Ideas > If you have questions you can contact the devs on IRC: irc.freenode.org #haikuThese projects are suggestions; you can come up with your own. Suggestions for other suitable projects are welcome.
Feel free to contact, or chat around on IRC: irc.freenode.org #netbsd #netbsd-code #pkgsrc
FreeBSD Jails with iocage
Introduction
Network setup These steps will: Set up the internal network. Enable the pf packet filter Configure pf pass internet traffic to and from the jail.FreeBSD jails allow users to run multiple, isolated instances of FreeBSD on a single server. Iocage simplifies the management of FreeBSD Jails. Following this tutorial, the jails will be configured to bind to an IP address on the jail host's internal network, and the host OS will pass traffic from the external network to the jail.
The jails will be managed with Iocage. Iocage uses ZFS properties to store configuration data for each jail, so a ZFS file system is required.
Run the following to configure the internal network and enable pf.PF is full featured firewall, and can do more than just pass traffic to an internal network. Refer to the PF documentation for additional configuration options.
Put the following in /etc/pf.confsysrc cloned_interfaces+="lo1" sysrc ifconfig_lo1="inet 192.0.2.1/24" sysrc pf_enable="YES"# Variables # ext_if should be set to the hosts external NIC ext_if = "vtnet0" jail_if = "lo1" jail_net = $jail_if:network # NAT allows the jails to access the external network nat on $ext_if from $jail_net to any -> ($ext_if) # Redirect traffic on port 80 to the web server jail # Add similar rules for additional jails rdr pass on $ext_if inet proto tcp to port 80 -> 192.0.2.10Reboot to activate the network changes
ZFS
Enable and start ZFS.The best way to use ZFS on a VPS is to attach block storage as a new disk.
If block storage is not available, you can optionally use a file as the ZFS device.
ZFS using Block storagesysrc zfs_enable="YES" service zfs startList the available disks.
If you are using a VPS, the block store will probably be the second disk.
Create a ZFS pool named jailstore.geom disk list
ZFS using a filezpool create jailstore /dev/vtbd1Create the ZFS file.
Create a ZFS pool named jailstore.dd if=/dev/zero of=/zfsfile bs=1M count=4096
Install iocage the easy wayzpool create jailstore /zfsfilepkg install py36-iocage
Install iocage the hard way Swap fileSkip to "Using iocage"
Smaller servers may not have enough RAM to build iocage. If needed, create a swap file and reboot.
Install some build dependenciesdd if=/dev/zero of=/swapfile bs=1M count=1024 echo 'swapfile="/swapfile"' >> /etc/rc.conf reboot
Building iocage requires the FreeBSD source.pkg install subversion python36 git-lite libgit2 py36-pip
Get the latest FreeBSD ports tree.svn checkout https://svn.freebsd.org/base/releng/11.1 /usr/src
build iocage.portsnap fetch portsnap extract
Using iocagecd /usr/ports/sysutils/iocage/ make installiocage activate jailstore iocage fetch iocage create -n www ip4_addr="lo1|192.0.2.10/24" -r 11.1-RELEASE iocage start www iocage console wwwOnce you have a shell inside the jail, install and start Apache.
pkg install apache24 sysrc apache24_enable="yes" service apache24 start
Multiple jails.Port 80 on the jail will now be accessible on the hosts IP address.
Install the new jail with the iocage create command , but use a different IP address Expose the new jail to the network by adding additional rules to pf.conf.Additional jails can be installed using the example above.
iXsystems
SNIA Persistent Memory Summit 2018 ReportNew FreeBSD Code of Conduct
The FreeBSD Project is inclusive. We want the FreeBSD Project to be a venue where people of all backgrounds can work together to make the best operating system, built by a strong community. These values extend beyond just development to all aspects of the Project. All those given recognition as members of the Project in whatever form are seen as ambassadors of the Project.
Diversity is a huge strength and is critical to the long term success of the Project. To that end we have a few ground rules that we ask people to adhere to. This code applies equally to everyone representing the FreeBSD Project in any way, from new members, to committers, to the core team itself. These rules are intended to ensure a safe, harassment-free environment for all and to ensure that everyone feels welcome both working within, and interacting with, the Project.
This document is not an exhaustive list of things that you should not do. Rather, consider it a guide to make it easier to enrich all of us and the technical communities in which we participate.
This code of conduct applies to all spaces used by the FreeBSD Project, including our mailing lists, IRC channels, and social media, both online and off. Anyone who is found to violate this code of conduct may be sanctioned or expelled from FreeBSD Project controlled spaces at the discretion of the FreeBSD Code of Conduct Committee.
Some FreeBSD Project spaces may have additional rules in place, which will be made clearly available to participants. Participants are responsible for knowing and abiding by these rules.
Harassment includes but is not limited to:
Comments that reinforce systemic oppression related to gender, gender identity and expression, sexual orientation, disability, mental illness, neurodiversity, physical appearance, body size, age, race, or religion. Unwelcome comments regarding a person's lifestyle choices and practices, including those related to food, health, parenting, drugs, and employment. Deliberate misgendering. Deliberate use of "dead" or rejected names. Gratuitous or off-topic sexual images or behaviour in spaces where they're not appropriate. Physical contact and simulated physical contact (e.g., textual descriptions like "hug" or "backrub") without consent or after a request to stop. Threats of violence. Incitement of violence towards any individual, including encouraging a person to commit suicide or to engage in self-harm. Deliberate intimidation. Stalking or following. Harassing photography or recording, including logging online activity for harassment purposes. Sustained disruption of discussion. Unwelcome sexual attention. Pattern of inappropriate social contact, such as requesting/assuming inappropriate levels of intimacy with others. Continued one-on-one communication after requests to cease. Deliberate "outing" of any private aspect of a person's identity without their consent except as necessary to protect vulnerable people from intentional abuse. Publication of non-harassing private communication without consent. Publication of non-harassing private communication with consent but in a way that intentionally misrepresents the communication (e.g., removes context that changes the meaning). Knowingly making harmful false claims about a person. ***Interview - Benno Rice - benno@freebsd.org / @jeamland
News Roundup
libhijack in PoC||GTFO 0x17!
Hijacking Your Free Beasties
Previous ResearchIn the land of red devils known as Beasties exists a system devoid of meaningful exploit mitigations. As we explore this vast land of opportunity, we will meet our ELFish friends, [p]tracing their very moves in order to hijack them. Since unprivileged process debugging is
enabled by default on FreeBSD, we can abuse PTrace to create anonymous memory mappings, inject code into them, and overwrite PLT/GOT entries. We will revive a tool called libhijack to make our nefarious activities of hijacking ELFs via PTrace relatively easy.Nothing presented here is technically new. However, this type of work has not been documented in this much detail, tying it all into one cohesive work. In Phrack 56, Silvio Cesare taught us ELF research enthusiasts how to hook the PLT/GOT. The Phrack 59 article on Runtime Process Infection briefly introduces the concept of injecting shared objects by injecting shellcode via PTrace that calls dlopen(). No other piece of research, however, has discovered the joys of forcing the application to create anonymous memory mappings in which to inject
Code.This is only part one of a series of planned articles that will follow libhijack's development. The end goal is to be able to anonymously inject shared objects. The libhijack project is maintained by the SoldierX community.
The Role of ELF > FreeBSD provides a nifty API for inspecting the entire virtual memory space of an application. The results returned from the API tells us the protection flags (readable, writable, executable) of each mapping. If FreeBSD provides such a rich API, why would we need to parse the ELF headers?All prior work injects code into the stack, the heap, or existing executable code. All three methods create issues on today's systems. On amd64 and arm64, the two architectures libhijack cares about, the stack is non-executable by default. jemalloc, the heap implementation
on FreeBSD, creates non-executable mappings. Obviously overwriting existing executable code destroys a part of the executable image.
The Future of libhijackPLT/GOT hijacking requires parsing ELF headers. One would not be able to find the PLT/GOT without iterating through the Process Headers to find the Dynamic Headers, eventually ending up with the DT_PLTGOT entry.
With FreeBSD's libprocstat API, we don't have a need for parsing ELF headers until we get to the PLT/GOT stage, but doing so early makes it easier for the attacker using libhijack
ConclusionWriting devious code in assembly is cumbersome. Assembly doesn't scale well to multiple architectures. Instead, we would like to write our devious code in C, compiling to a shared object that gets injected anonymously. This requires writing a remote RTLD within libhijack and
is in progress. Writing a remote RTLD will take a while as doing so is not an easy task.Additionally, creation of a general-purpose helper library that gets injected would be helpful. It could aid in PLT/GOT redirection attacks, possibly storing the addresses of functions we've previously hijacked. This work is dependent on the remote RTLD.
libhijack currently lacks documentation. Once the ABI and API stabilize, formal documentation will be written.
libhijack can be found at https://github.com/SoldierX/libhijackUsing libhijack, we can easily create anonymous memory mappings, inject into them arbitrary code, and hijack the PLT/GOT on FreeBSD. On HardenedBSD, a hardened derivative of FreeBSD, libhijack is fully mitigated through PaX NOEXEC.
We've demonstrated that wrapper-style Capsicum is ineffective on FreeBSD. Through the use of libhijack, we emulate a control flow hijack in which the application is forced to call sandbox_open and fdlopen on the resulting file descriptor.
Further work to support anonymous injection of full shared objects, along with their dependencies, will be supported in the future. Imagine injecting libpcap into Apache to sniff traffic whenever "GET /pcap" is sent.
In order to prevent abuse of PTrace, FreeBSD should set the security.bsd.unprivileged_proc_debug to 0 by default. In order to prevent process manipulation, FreeBSD should implement PaX NOEXEC.
Introduction to POSIX shell
Why use POSIX shell?What the heck is the POSIX shell anyway? Well, the POSIX (the Portable Operating System Interface) shell is the standard Unix shell - standard meaning it was formally defined and shipped in a published standard. This makes shell scripts written for it portable, something no other shell can lay claim to. The POSIX shell is basically a formalized version of the venerable Bourne shell, and on your system it lives at /bin/sh, unless you’re one of the unlucky masses for whom this is a symlink to bash.
How do I use POSIX shell?The “Bourne Again shell”, aka bash, is not standardized. Its grammar, features, and behavior aren’t formally written up anywhere, and only one implementation of bash exists. Without a standard, bash is defined by its implementation. POSIX shell, on the other hand, has many competing implementations on many different operating systems - all of which are compatible with each other because they conform to the standard.
Any shell that utilizes features specific to Bash are not portable, which means you cannot take them with you to any other system. Many Linux-based systems do not use Bash or GNU coreutils. Outside of Linux, pretty much everyone but Hurd does not ship GNU tools, including bash1. On any of these systems, scripts using “bashisms” will not work.
This is bad if your users wish to utilize your software anywhere other than GNU/Linux. If your build tooling utilizes bashisms, your software will not build on anything but GNU/Linux. If you ship runtime scripts that use bashisms, your software will not run on anything but GNU/Linux. The case for sticking to POSIX shell in shipping software is compelling, but I argue that you should stick to POSIX shell for your personal scripts, too. You might not care now, but when you feel like flirting with other Unicies you’ll thank me when all of your scripts work.
One place where POSIX shell does not shine is for interactive use - a place where I think bash sucks, too. Any shell you want to use for your day-to-day command line work is okay in my book. I use fish. Use whatever you like interactively, but stick to POSIX sh for your scripts.
[[ condition ]] does not work; use [ condition ] Arrays do not work; use IFS Local variables do not work; use a subshellAt the top of your scripts, put #!/bin/sh. You don’t have to worry about using env here like you might have been trained to do with bash: /bin/sh is the standardized location for the POSIX shell, and any standards-conforming system will either put it there or make your script work anyway.
The next step is to avoid bashisms. There are many, but here are a few that might trip you up:
Using standard coreutilsThe easiest way to learn about POSIX shell is to read the standard - it’s not too dry and shorter than you think.
Some tips for using short flags: They can be collapsed: cmd -a -b -c is equivalent to cmd -abc If they take additional arguments, either a space or no separation is acceptable: cmd -f"hello world" or cmd -f "hello world"The last step to writing portable scripts is to use portable tools. Your system may have GNU coreutils installed, which provides tools like grep and cut. Unfortunately, GNU has extended these tools with its own non-portable flags and tools. It’s important that you avoid these.
One dead giveaway of a non-portable flag is long flags, e.g. grep --file=FILE as opposed to grep -f. The POSIX standard only defines the getopt function - not the proprietary GNU getopt_long function that’s used to interpret long options. As a result, no long flags are standardized. You might worry that this will make your scripts difficult to understand, but I think that on the whole it will not. Shell scripts are already pretty alien and require some knowledge to understand. Is knowledge of what the magic word grep means much different from knowledge of what grep -E means?
I also like that short flags allow you to make more concise command lines. Which is better: ps --all --format=user --without-tty, or ps -aux? If you are inclined to think the former, do you also prefer function(a, b, c) { return a + b + c; } over (a, b, c) => a + b + c? Conciseness matters, and POSIX shell supports comments if necessary!
A good reference for learning about standardized commands is, once again, the standard. From this page, search for the command you want, or navigate through “Shell & Utilities” -> “Utilities” for a list. If you have man-pages installed, you will also find POSIX man pages installed on your system with the p postfix, such as man 1p grep. Note: at the time of writing, the POSIX man pages do not use dashes if your locale is UTF-8, which makes searching for flags with / difficult. Use env LC_ALL=POSIX man 1p grep if you need to search for flags, and I’ll speak to the maintainer of man-pages about this.
FreeBSD Broadcom Wi-Fi Improvements
Introduction
A Brief Background on Broadcom Wi-Fi HardwareSince 2015, I've been working on improving FreeBSD support for Broadcom Wi-Fi devices and SoCs, including authoring the bhnd(4) driver family, which provides a unified bus and driver programming interface for these devices.
First committed in early 2016, bhnd(4) allowed us to quickly bring up FreeBSD/MIPS on Broadcom SoCs, but it has taken much longer to implement the full set of features required to support modern Broadcom SoftMAC Wi-Fi hardware.
Thanks to the generosity of the FreeBSD Foundation, I've recently finished implementing the necessary improvements to the bhnd(4) driver family. With these changes in place, I was finally able to port the existing bwn(4) Broadcom SoftMAC Wi-Fi driver to the bhnd(4) bus, and implement initial support for the BCM43224 and BCM43225 chipsets, with additional hardware support to be forthcoming.
Now that my efforts on FreeBSD/Broadcom Wi-Fi support have progressed far enough to be generally useful, I wanted to take some time to provide a brief overview of Broadcom's Wi-Fi hardware, and explain how my work provides a foundation for further FreeBSD Broadcom Wi-Fi/SoC improvements.
BHND devices utilize a common set of Broadcom IP cores (or "functional blocks") connected via one of two on-chip bus architectures: Hardware designed prior to 2009 used Broadcom’s “SSB” backplane architecture, based on Sonics Silicon’s interconnect IP. Subsequent hardware adopted Broadcom’s “BCMA” backplane, based on ARM’s AMBA IP. The IP cores used in earlier SSB-based devices were adapted for compatibility with the new backplane.Broadcom's Wi-Fi devices are members of the Broadcom Home Networking Division (BHND) device family; other BHND devices include MIPS/ARM SoCs (including Wi-Fi SoCs commonly found in consumer access points), as well as a large variety of related networking hardware.
Host access to SoC address space is provided via a set of register windows (e.g., a set of configurable windows into SoC address space mapped via PCI BARs) DMA is supported by the bridge core’s sparse mapping of host address space into the backplane address space. These address regions may be used as a target for the on-chip DMA engines. Any backplane interrupt vectors routed to the bridge core may be mapped by the bridge to hoWhen BHND hardware is used in a PCI Wi-Fi card, or a SDIO Wi-Fi module, the device's dual-mode peripheral controller is configured to operate as an endpoint device on the host's peripheral bus, bridging access to the SoC hardware:
Wed, 21 Feb 2018 - 1h 43min - 833 - 233: High on ZFS
We explain the physics behind ZFS, DTrace switching to the GPL, Emacs debugging, syncookies coming to PF & FreeBSD’s history on EC2.
This episode was brought to you by
Headlines
128 bit storage: Are you high?
For people who have heard about ZFS boiling oceans and wonder where that is coming from, we dug out this old piece from 2004 on the blog of ZFS co-creator Jeff Bonwick, originally from the Sun website.
Best part of all: you don’t have to understand any of this to use ZFS. Rest assured that you won’t hit any limits with that filesystem for a long time. You still have to buy bigger disks over time, though... ***64 bits would have been plenty ... but then you can't talk out of your ass about boiling oceans then, can you?
Well, it's a fair question. Why did we make ZFS a 128-bit storage system? What on earth made us think it's necessary? And how do we know it's sufficient? Let's start with the easy one: how do we know it's necessary?
Some customers already have datasets on the order of a petabyte, or 250 bytes. Thus the 64-bit capacity limit of 264 bytes is only 14 doublings away. Moore's Law for storage predicts that capacity will continue to double every 9-12 months, which means we'll start to hit the 64-bit limit in about a decade. Storage systems tend to live for several decades, so it would be foolish to create a new one without anticipating the needs that will surely arise within its projected lifetime.
If 64 bits isn't enough, the next logical step is 128 bits. That's enough to survive Moore's Law until I'm dead, and after that, it's not my problem. But it does raise the question: what are the theoretical limits to storage capacity?
Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 1051 operations per second on at most 1031 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2128 blocks = 2137 bytes = 2140 bits; therefore the minimum mass required to hold the bits would be (2140 bits) / (1031 bits/kg) = 136 billion kg.
That's a lot of gear.
To operate at the 1031 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc2, the rest energy of 136 billion kg is 1.2x1028 J. The mass of the oceans is about 1.4x1021 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling.
The latent heat of vaporization adds another 2 million J/kg.
Thus the energy required to boil the oceans is about 2.4x106 J/kg * 1.4x1021 kg =
3.4x1027 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans.dtrace for Linux, Oracle relicenses dtrace
At Fosdem we had a talk on dtrace for linux in the Debugging Tools devroom.
Not explicitly mentioned in that talk, but certainly the most exciting thing, is that Oracle is doing a proper linux kernel port:
commit e1744f50ee9bc1978d41db7cc93bcf30687853e6 Author: Tomas Jedlicka <tomas.jedlicka@oracle.com> Date: Tue Aug 1 09:15:44 2017 -0400 dtrace: Integrate DTrace Modules into kernel proper This changeset integrates DTrace module sources into the main kernel source tree under the GPLv2 license. Sources have been moved to appropriate locations in the kernel tree.That is right, dtrace dropped the CDDL and switched to the GPL!
The user space code dtrace-utils and libdtrace-ctf (a combination of GPLv2 and UPL) can be found on the DTrace Project Source Control page. The NEWS file mentions the license switch (and that it is build upon elfutils, which I personally was pleased to find out).
The kernel sources (GPLv2+ for the core kernel and UPL for the uapi) are slightly harder to find because they are inside the uek kernel source tree, but following the above commit you can easily get at the whole linux kernel dtrace directory.
The UPL is the Universal Permissive License, which according to the FSF is a lax, non-copyleft license that is compatible with the GNU GPL.
Thank you Oracle for making everyone’s life easier by waving your magic relicensing wand!
Now there is lots of hard work to do to actually properly integrate this. And I am sure there are a lot of technical hurdles when trying to get this upstreamed into the mainline kernel. But that is just hard work. Which we can now start collaborating on in earnest.
Like systemtap and the Dynamic Probes (dprobes) before it, dtrace is a whole system observability tool combining tracing, profiling and probing/debugging techniques. Something the upstream linux kernel hackers don’t always appreciate when presented as one large system. They prefer having separate small tweaks for tracing, profiling and probing which are mostly separate from each other. It took years for the various hooks, kprobes, uprobes, markers, etc. from systemtap (and other systems) to get upstream. But these days they are. And there is now even a byte code interpreter (eBPF) in the mainline kernel as originally envisioned by dprobes, which systemtap can now target through stapbpf. So with all those techniques now available in the linux kernel it will be exciting to see if dtrace for linux can unite them all.
Debugging Emacs or: How I Learned to Stop Worrying and Love DTrace
Bryan Cantrill, DTrace, and FreeBSDFor some time Elfeed was experiencing a strange, spurious failure. Every so often users were seeing an error (spoiler warning) when updating feeds: “error in process sentinel: Search failed.” If you use Elfeed, you might have even seen this yourself. From the surface it appeared that curl, tasked with the responsibility for downloading feed data, was producing incomplete output despite reporting a successful run. Since the run was successful, Elfeed assumed certain data was in curl’s output buffer, but, since it wasn’t, it failed hard.
Unfortunately this issue was not reproducible. Manually running curl outside of Emacs never revealed any issues. Asking Elfeed to retry fetching the feeds would work fine. The issue would only randomly rear its head when Elfeed was fetching many feeds in parallel, under stress. By the time the error was discovered, the curl process had exited and vital debugging information was lost. Considering that this was likely to be a bug in Emacs itself, there really wasn’t a reliable way to capture the necessary debugging information from within Emacs Lisp. And, indeed, this later proved to be the case.
A quick-and-dirty work around is to use condition-case to catch and swallow the error. When the bizarre issue shows up, rather than fail badly in front of the user, Elfeed could attempt to swallow the error — assuming it can be reliably detected — and treat the fetch as simply a failure. That didn’t sit comfortably with me. Elfeed had done its due diligence checking for errors already. Someone was lying to Elfeed, and I intended to catch them with their pants on fire. Someday.
I’d just need to witness the bug on one of my own machines. Elfeed is part of my daily routine, so surely I’d have to experience this issue myself someday. My plan was, should that day come, to run a modified Elfeed, instrumented to capture extra data. I would have also routinely run Emacs under GDB so that I could inspect the failure more deeply.
For now I just had to wait to hunt that zebra.
What is DTrace?Over the holidays I re-discovered Bryan Cantrill, a systems software engineer who worked for Sun between 1996 and 2010, and is most well known for DTrace. My first exposure to him was in a BSD Now interview in 2015. I had re-watched that interview and decided there was a lot more I had to learn from him. He’s become a personal hero to me. So I scoured the internet for more of his writing and talks.
Some interesting operating system technology came out of Sun during its final 15 or so years — most notably DTrace and ZFS — and Bryan speaks about it passionately. Almost as a matter of luck, most of it survived the Oracle acquisition thanks to Sun releasing it as open source in just the nick of time. Otherwise it would have been lost forever. The scattered ex-Sun employees, still passionate about their prior work at Sun, along with some of their old customers have since picked up the pieces and kept going as a community under the name illumos. It’s like an open source flotilla.
Naturally I wanted to get my hands on this stuff to try it out for myself. Is it really as good as they say? Normally I stick to Linux, but it (generally) doesn’t have these Sun technologies available. The main reason is license incompatibility. Sun released its code under the CDDL, which is incompatible with the GPL. Ubuntu does infamously include ZFS, but other distributions are unwilling to take that risk. Porting DTrace is a serious undertaking since it’s got its fingers throughout the kernel, which also makes the licensing issues even more complicated.
Linux has a reputation for Not Invented Here (NIH) syndrome, and these licensing issues certainly contribute to that. Rather than adopt ZFS and DTrace, they’ve been reinvented from scratch: btrfs instead of ZFS, and a slew of partial options instead of DTrace. Normally I’m most interested in system call tracing, and my go to is strace, though it certainly has its limitations — including this situation of debugging curl under Emacs. Another famous example of NIH is Linux’s epoll(2), which is a broken version of BSD kqueue(2).
So, if I want to try these for myself, I’ll need to install a different operating system. I’ve dabbled with OmniOS, an OS built on illumos, in virtual machines, using it as an alien environment to test some of my software (e.g. enchive). OmniOS has a philosophy called Keep Your Software To Yourself (KYSTY), which is really just code for “we don’t do packaging.” Honestly, you can’t blame them since they’re a tiny community. The best solution to this is probably pkgsrc, which is essentially a universal packaging system. Otherwise you’re on your own.
There’s also openindiana, which is a more friendly desktop-oriented illumos distribution. Still, the short of it is that you’re very much on your own when things don’t work. The situation is like running Linux a couple decades ago, when it was still difficult to do.
If you’re interested in trying DTrace, the easiest option these days is probably FreeBSD. It’s got a big, active community, thorough documentation, and a huge selection of packages. Its license (the BSD license, duh) is compatible with the CDDL, so both ZFS and DTrace have been ported to FreeBSD.
Bryan Cantrill: Talks I have given ***I’ve done all this talking but haven’t yet described what DTrace really is. I won’t pretend to write my own tutorial, but I’ll provide enough information to follow along. DTrace is a tracing framework for debugging production systems in real time, both for the kernel and for applications. The “production systems” part means it’s stable and safe — using DTrace won’t put your system at risk of crashing or damaging data. The “real time” part means it has little impact on performance. You can use DTrace on live, active systems with little impact. Both of these core design principles are vital for troubleshooting those really tricky bugs that only show up in production.
There are DTrace probes scattered all throughout the system: on system calls, scheduler events, networking events, process events, signals, virtual memory events, etc. Using a specialized language called D (unrelated to the general purpose programming language D), you can dynamically add behavior at these instrumentation points. Generally the behavior is to capture information, but it can also manipulate the event being traced.
Each probe is fully identified by a 4-tuple delimited by colons: provider, module, function, and probe name. An empty element denotes a sort of wildcard. For example, syscall::open:entry is a probe at the beginning (i.e. “entry”) of open(2). syscall:::entry matches all system call entry probes.
Unlike strace on Linux which monitors a specific process, DTrace applies to the entire system when active. To run curl under strace from Emacs, I’d have to modify Emacs’ behavior to do so. With DTrace I can instrument every curl process without making a single change to Emacs, and with negligible impact to Emacs. That’s a big deal.
So, when it comes to this Elfeed issue, FreeBSD is much better poised for debugging the problem. All I have to do is catch it in the act. However, it’s been months since that bug report and I’m not really making this connection yet. I’m just hoping I eventually find an interesting problem where I can apply DTrace.
News Roundup
a2k18 Hackathon preview: Syncookies coming to PF
As you may have heard, the a2k18 hackathon is in progress. As can be seen from the commit messages, several items of goodness are being worked on.
One eagerly anticipated item is the arrival of TCP syncookies (read: another important tool in your anti-DDoS toolset) in PF. Henning Brauer (henning@) added the code in a series of commits on February 6th, 2018, with this one containing the explanation:syncookies for pf. when syncookies are on, pf will blindly answer each and every SYN with a syncookie-SYNACK. Upon reception of the ACK completing the 3WHS, pf will reconstruct the original SYN, shove it through pf_test, where state will be created if the ruleset permits it. Then massage the freshly created state (we won't see the SYNACK), set up the sequence number modulator, and call into the existing synproxy code to start the 3WHS with the backend host. Add an - somewhat basic for now - adaptive mode where syncookies get enabled if a certain percentage of the state table is filled up with half-open tcp connections. This makes pf firewalls resilient against large synflood attacks. syncookies are off by default until we gained more experience, considered experimental for now. see http://bulabula.org/papers/2017/bsdcan/ for more details. joint work with sashan@, widely discussed and with lots of input by manyThe first release to have this feature available will probably be the upcoming OpenBSD 6.3 if a sufficient number of people test this in their setups (hint, hint). More info is likely to emerge soon in post-hackathon writeups, so watch this space!
[Pale Moon]
A Perfect example of how not to approach OS developers/packagers Removed from OpenBSD Ports due to Licensing Issues FreeBSD Palemoon branding violation Mightnight BSD's response ***FreeBSD EC2 History
August 25, 2006: Amazon EC2 launches. It supports a single version of Ubuntu Linux; FreeBSD is not available. December 13, 2010: I manage to get FreeBSD running on EC2 t1.micro instances. March 22, 2011: I manage to get FreeBSD running on EC2 "cluster compute" instances. July 8, 2011: I get FreeBSD 8.2 running on all 64-bit EC2 instance types, by marking it as "Windows" in order to get access to Xen/HVM virtualization. (Unfortunately this meant that users had to pay the higher "Windows" hourly pricing.) January 16, 2012: I get FreeBSD 9.0 running on 32-bit EC2 instances via the same "defenestration" trick. (Again, paying the "Windows" prices.) August 16, 2012: I move the FreeBSD rc.d scripts which handle "EC2" functionality (e.g., logging SSH host keys to the console) into the FreeBSD ports tree. October 7, 2012: I rework the build process for FreeBSD 9.1-RC1 and later to use "world" bits extracted from the release ISOs; only the kernel is custom-built. Also, the default SSH user changes from "root" to "ec2-user". October 31, 2012: Amazon launches the "M3" family of instances, which support Xen/HVM without FreeBSD needing to pay the "Windows" tax. November 21, 2012: I get FreeBSD added to the AWS Marketplace. October 2, 2013: I finish merging kernel patches into the FreeBSD base system, and rework the AMI build (again) so that FreeBSD 10.0-ALPHA4 and later use bits extracted from the release ISOs for the entire system (world + kernel). FreeBSD Update can now be used for updating everything (because now FreeBSD/EC2 uses a GENERIC kernel). October 27, 2013: I add code to EC2 images so that FreeBSD 10.0-BETA2 and later AMIs will run FreeBSD Update when they first boot in order to download and install any critical updates. December 1, 2013: I add code to EC2 images so that FreeBSD 10.0-BETA4 and later AMIs bootstrap the pkg tool and install packages at boot time (by default, the "awscli" package). December 9, 2013: I add configinit to FreeBSD 10.0-RC1 and later to allow systems to be easily configured via EC2 user-data. July 1, 2014: Amazon launches the "T2" family of instances; now the most modern family for every type of EC2 instance (regular, high-memory, high-CPU, high-I/O, burstable) supports HVM and there should no longer be any need for FreeBSD users to pay the "Windows tax". November 24, 2014: I add code to FreeBSD 10.2 and later to automatically resize their root filesystems when they first boot; this means that a larger root disk can be specified at instance launch time and everything will work as expected. April 1, 2015: I integrate the FreeBSD/EC2 build process into the FreeBSD release building process; FreeBSD 10.2-BETA1 and later AMIs are built by the FreeBSD release engineering team. January 12, 2016: I enable Intel 82599-based "first generation EC2 Enhanced Networking" in FreeBSD 11.0 and later. June 9, 2016: I enable the new EC2 VGA console functionality in FreeBSD 11.0 and later. (The old serial console also continues to work.) June 24, 2016: Intel 82599-based Enhanced Networking works reliably in FreeBSD 11.0 and later thanks to discovering and working around a Xen bug. June 29, 2016: I improve throughput on Xen blkfront devices (/dev/xbd*) by enabling indirect segment I/Os in FreeBSD 10.4 and later. (I wrote this functionality in July 2015, but left it disabled by default a first because a bug in EC2 caused it to hurt performance on some instances.) July 7, 2016: I fix a bug in FreeBSD's virtual memory initialization in order to allow it to support boot with 128 CPUs; aka. FreeBSD 11.0 and later support the EC2 x1.32xlarge instance type. January 26, 2017: I change the default configuration in FreeBSD 11.1 and later to support EC2's IPv6 networking setup out of the box (once you flip all of the necessary switches to enable IPv6 in EC2 itself). May 20, 2017: In collaboration with Rick Macklem, I make FreeBSD 11.1 and later compatible with the Amazon "Elastic File System" (aka. NFSv4-as-a-service) via the newly added "oneopenown" mount option (and lots of bug fixes). May 25, 2017: I enable support for the Amazon "Elastic Network Adapter" in FreeBSD 11.1 and later. (The vast majority of the work — porting the driver code — was done by Semihalf with sponsorship from Amazon.)A couple years ago Jeff Barr published a blog post with a timeline of EC2 instances. I thought at the time that I should write up a timeline of the FreeBSD/EC2 platform, but I didn't get around to it; but last week, as I prepared to ask for sponsorship for my work I decided that it was time to sit down and collect together the long history of how the platform has evolved and improved over the years.
Normally I don't edit blog posts after publishing them (with the exception of occasional typographical corrections), but I do plan on keeping this post up to date with future developments.December 5, 2017: I change the default configuration in FreeBSD 11.2 and later to make use of the Amazon Time Sync Service (aka. NTP-as-a-service).
The current status
The upcoming FreeBSD release (11.2) supports: IPv6, Enhanced Networking (both generations), Amazon Elastic
Wed, 14 Feb 2018 - 1h 50min - 832 - 232: FOSDEM 2018
We talk about our recent trip to FOSDEM, we discuss the pros and cons of permissive licensing, cover the installation of OpenBSD on a dedibox with full-disk encryption, the new Lumina guide repository, and we explain ZFS vs. OpenZFS.
This episode was brought to you by
Headlines
[FOSDEM Trip report]
Your BSDNow hosts were both at FOSDEM in Brussels, Belgium over the weekend. On the friday before FOSDEM, we held a FreeBSD devsummit (3rd consecutive year), sponsored by the FreeBSD Foundation and organized by Benedict (with the help from Kristof Provost, who did it in previous years but could not make it this year). We had 21 people attend, a good mixture of FreeBSD committers (mostly ports) and guests. After introductions, we collected topics and discussed various topics, including a new plan for a future FreeBSD release roadmap (more frequent releases, so that features from HEAD can be tried out earlier in RELEASES). The devsummit concluded with a nice dinner in a nearby restaurant. On Saturday, first day of FOSDEM, we set up the FreeBSD Foundation table with flyers, stickers, FreeBSD Journal print editions, and a small RPI 3 demo system that Deb Goodkin brought. Our table was located next to the Illumos table like last year. This allowed us to continue the good relationship that we have with the Illumos people and Allan helped a little bit getting bhyve to run on Illumos with UEFI. Meanwhile, our table was visited by a lot of people who would ask questions about FreeBSD, take info material, or talk about their use cases. We were busy refilling the table throughout the day and luckily, we had many helpers at the table. Some items we had ran out in the early afternoon, an indicator of how popular they were. Saturday also featured a BSD devroom, organized by Rodrigo Osorio. You can find the list of talks and the recordings on the BSD Devroom schedule. The room was very crowded and popular. Deb Goodkin gave the opening talk with an overview of what the Foundation is doing to change the world. Other speakers from various BSD projects presented their talks after that with a range of topics. Among them, Allan gave his talk about ZFS: Advanced Integration, while Benedict presented his Reflections on Teaching a Unix Class With FreeBSD. Sunday was just as busy on the FreeBSD table as Saturday and we finally ran out of stickers and some other goodies. We were happy with the results of the two days. Some very interesting conversations at the table about FreeBSD took place, some of which were going to follow up afterwards. Check out the FOSDEM schedule as many talk recordings are already available, and especially the ones from the BSD devroom if you could not attend the conference. We would like to thank everyone who attended the FreeBSD devsummit, who helped out at the FreeBSD table and organized the BSD devroom. Also, thanks to all the speakers, organizers, and helping hands making FOSDEM another success this year. ***NetBSD kernel wscons IOCTL vulnerable bug class
I discovered this bug class during the InfoSect public code review session we ran looking specifically at the NetBSD kernel. I found a couple of these bugs and then after the session was complete, I went back and realised the same bug was scattered in other drivers. In total, 17 instances of this vulnerability and its variants were discovered.
In all fairness, I came across this bug class during my kernel audits in 2002 and most instances were patched. It just seems there are more bugs now in NetBSD while OpenBSD and FreeBSD have practically eliminated them.See slide 41 in http://www.blackhat.com/presentations/bh-usa-03/bh-us-03-cesare.pdf for exactly the same bug (class) 16 years ago.
The format of the this blog post is as follows:
Introduction
Example of the Bug Class
How to Fix
How to Detect Automatically with Coccinelle
More Bugs
Conclusion
These source files had bugs
./dev/tc/tfb.c ./dev/ic/bt485.c ./dev/pci/radeonfb.c ./dev/ic/sti.c ./dev/sbus/tcx.c ./dev/tc/mfb.c ./dev/tc/sfb.c ./dev/tc/stic.c ./dev/tc/cfb.c ./dev/tc/xcfb.c ./dev/tc/sfbplus.c ./arch/arm/allwinner/awin_debe.c ./arch/arm/iomd/vidcvideo.c ./arch/pmax/ibus/pm.c ./dev/ic/igfsb.c ./dev/ic/bt463.c ./arch/luna68k/dev/lunafb.c
The patches to fix these issues are in NetBSD: https://mail-index.netbsd.org/source-changes/2018/01/24/msg091428.htmlReporting of the bugs was easy. In less than a week from reporting the specific instances of each bug, patches were committed into the mainline kernel. Thanks to Luke Mewburn from NetBSD for coming to the code review session at InfoSect and coordinating with the NetBSD security team.
"Permissive licensing is wrong! Is it?
A few weeks ago Ive been attacked by some GNU zealots on a German tech site after speaking in favor of permissive licenses. Unfortunately a discussion was not possible there because that would require the will to actually communicate instead of simply accusing the other side of vile motives. Since I actually do care about this topic and a reader asked for a post about it in comments a while ago, here we go.
This first part tries to sum up the most important things around the topic. I deliberately aim for an objective overview that tries not to be one-sided. The second part will then contain my points in defence of permissive licensing.
Why license software at all?
Licenses exist for reasons of protection. If youre the author/inventor of some software, a story or whatever product, you get to decide what to do with it. You can keep it for yourself or you can give it away. If you decide for the latter, you have to decide who may use it and in which way(s). In case you intend to give it to a (potentially) large group of people, you may not want to be asked for permission to xyz by everybody. Thats when you decide to write a license which states what you are allowing and explicitly disallowing.
Most of the well-known commercial licenses focus on what youre not allowed to do (usually things like copying, disassembling, etc.). Open source licenses on the other hand are meant to grant the user rights (e.g. the right to distribute) while reserving some rights or only giving permission under certain conditions and they usually make you claim responsibility for using the software. For these reasons licenses can actually be a good thing!
If you got an unlicensed piece of code, youre not legally allowed to do anything with it without getting the authors permission first. And even if you got that permission, your project would be risky, since the author can withdraw it later. A proper license protects both parties. The author doesnt get his mail account full of email asking for permission, hes save from legal trouble if his code breaks anything for you and at the same time you have legal certainty when you decide to put the code to long-term use.Permissive vs. Copyleft (in a nutshell)
The GNU Public license (GPL)In short terms, permissive licensing usually goes like this: Here you are, have fun. Oh, and dont sue me if it does something else than what you expect! Yes, its that easy and theres little to dispute over.
Copyleft on the other side sounds like this (if you ask somebody in favor of Copyleft): Sure, you can use it, its free. Just keep it free, ok?. Also quite simple. And not too bad, eh? Other people however read the same thing like this: Yes, youre free to use it. Just read these ten pages of legalese and be dead certain that you comply. If you got something wrong, we will absolutely make you regret it.The most popular copyleft license in use is the GPL (in various versions). It got more and more complex with each version and to be fair, it had to, because it was necessary to react to new threats and loop holes that were found later. The GNU project states that they are committed to protect what they call the four freedoms of free software:
the freedom to use the software for any purpose the freedom to change the software to suit your needs the freedom to share the software with your friends and neighbors the freedom to share the changes you make
The BSD licenseThese are freedoms that every supporter of open source software should be able to agree with. So whats the deal with all the hostility and fighting between the two camps? Lets take a look at a permissive license, too.
Whats the problem?Unlike the GPL, the BSD family of licenses begun with a rather simple license that span four rules (original BSD license). It was later revised and reduced to three (modified BSD license). And the modern BSD license that e.g. FreeBSD uses is even just two (simplified BSD license).
Did you read the GPLv3 that I linked to above? If you are using GPLd code you really should. In case you dont feel like reading all of it, at least take a look and grasp how long that text is. Now compare it to the complete modern BSD license.
Whos right?There are essentially two problems that cause all the trouble. The first one is the question of what should be subject to the freedom that were talking about. And closely related, the second one is where that freedom needs to end.
Ironically both camps claim that freedom is the one important thing and it must not be restricted. The GPL is meant to protect the freedom of the software and enforces the availability of the source code, hence limiting the freedom of actual persons. BSD on the other hand is meant to protect the freedom of human beings who should be able to use the software as they see fit even if that means closing down former open source code!
The GNU camp taunts permissive licenses as being lax for not providing the protection that they want. The other camp points out that the GPL is a complex monster and that it is virulent in nature: Since its very strict in a lot of areas, its incompatible with many other licenses. This makes it complicated to mix GPL and non-GPL code and in the cases where its legally possible, the GPLs terms will take precedence and necessarily be in effect for the whole combined work.
Permissive licensing is wrong? No its not!That totally depends on what you want to achieve. There are pros and cons to both and in fact were only looking at the big picture here. Theres also e.g. the Apache license which is often deemed as kind of middle ground. Then you may want to consider the difference between weak (e.g. LGPL) as well as strong copyleft (GPL). Licensing is a potentially huge topic. But lets keep it simple here because the exact details are actually not necessary to understand the essence of our topic.
In the next post Ill present my stance on why permissive licensing is a good thing and copyleft is more problematic than many people may think.
The previous post gave a short introduction into the topic of software licenses, focusing on the GPL vs. BSD discussion. This one is basically my response to some typical arguments Ive seen from people who seem to loathe permissive licensing. Ill write this in dialog style, hoping that this makes it a little lighter to read.
Roundup
Install OpenBSD on dedibox with full-disk encryption
TL;DR:
Step #0: choosing your serverI run several "dedibox" servers at online.net, all powered by OpenBSD. OpenBSD is not officially supported so you have to work-around. Running full-disk encrypted OpenBSD there is a piece of cake. As a bonus, my first steps within a brand new booted machine ;-)
Step #1: boot to rescue mode Step #2: boot to the installer Step #3: prepare softraid Step #4: reboot to encrypted OpenBSD system Bonus: further tightening your system enable doas disable the root account update system with syspatch add my ssh public key to my ~/.ssh/authorized_keys disable password authentication within ssh reboot so you boot on a brand new up-to-date system with latest stable kernel VOILA !OpenBSD is not officially supported, I cant guarantee that this will work for you on any kind of server online.net provides, however Ive been running https://poolp.org on OpenBSD there since 2008, only switching machines as they were getting a bit old and new offers came up.
Currently, Im running two SC 2016 (SATA) and one XC 2016 (SSD) boxes, all three running OpenBSD reliably ever since I installed them.
Recently Ive been willing to reinstall the XC one after I did some experiments that turned it into a FrankenBSD, so this was the right occasion to document how I do it for future references.
I wrote an article similar to this a few years ago relying on qemu to install to the disk, since then online.net provided access to a virtual serial console accessed within the browser, making it much more convenient to install without the qemu indirection which hid the NIC devices and disks duid and required tricks.
The method I currently use is a mix and adaptation from the techniques described in https://www.2f30.org/guides/openbsd-dedibox.html to boot the installer, and the technique described in https://geekyschmidt.com/2011/01/19/configuring-openbsd-softraid-fo-encryption.html to setup the crypto slice.
January 2018 Development Projects Update
Spectre and Meltdown in FreeBSDIssues affecting most CPUs used in servers, desktops, laptops, and mobile devices are in the news. These hardware vulnerabilities, known by the code-names Meltdown and Spectre, allow malicious programs to read data to which they should not have access. This potentially includes credentials, cryptographic material, or other secrets. They were originally identified by a researcher from Googles Project Zero, and were also independently discovered by researchers and academics from Cyberus Technology, Graz University of Technology, the University of Pennsylvania, the University of Maryland, Rambus, the University of Adelaide and Data61.
These vulnerabilities affect many CPU architectures supported by FreeBSD, but the 64-bit x86 family of processors from Intel and AMD are the most widely used, and are a high priority for software changes to mitigate the effects of Meltdown and Spectre. In particular, the Meltdown issue affects Intel CPUs and may be used to extract secret data from the running kernel, and therefore, is the most important issue to address.
The FreeBSD Foundation collaborates with Intel, and under this relationship participated in a briefing to understand the details of these issues and plan the mitigations to be applied to the x86 architectures supported by FreeBSD. We also made arrangements to have FreeBSDs security officer join me in the briefing. It is through the generous support of the Foundations donors that we are able to dedicate resources to focus on these issues on demand as they arise.
Foundation staff member Konstantin (Kostik) Belousov is an expert on FreeBSDs Virtual Memory (VM) system as well as low-level x86 details, and is developing the x86 kernel mitigations for FreeBSD.
The mitigation for Meltdown is known as Page Table Isolation (PTI). Kostik created a PTI implementation which was initially committed in mid-January and is available in the FreeBSD-CURRENT development repository. This is the same approach used by the Linux kernel to mitigate Meltdown.
One of the drawbacks of the PTI mitigation is that it incurs a performance regression. Kostik recently reworked FreeBSDs use of Process-Context Identifiers (PCID) in order to regain some of the performance loss incurred by PTI. This change is also now available in FreeBSD-CURRENT.
The issue known as Spectre comes in two variants, and variant 2 is the more troubling and pressing one. It may be mitigated in one of two ways: by using a technique called retpoline in the compiler, or by making use of a CPU feature introduced in a processor microcode update. Both options are under active development. Kostiks change to implement the CPU-based mitigation is currently in review. Unfortunately, it introduces a significant performance penalty and alternatives are preferred, if available.
For most cases, the compiler-based retpoline mitigation is likely to be the chosen mitigation. Having switched to the Clang compiler for the base system and most of the ports collection some years ago, FreeBSD is well-positioned to deploy Clang-based mitigations. FreeBSD developer Dimitry Andric is spearheading the update of Clang/LLVM in FreeBSD to version 6.0 in anticipation of its official release; FreeBSD-CURRENT now includes an interim snapshot. I have been assisting with the import, particularly with respect to LLVMs lld linker, and will support the integration of retpoline. This support is expected to be merged into FreeBSD in the coming weeks.
The Foundations co-op students have also participated in the response to these vulnerabilities. Mitchell Horne developed the patch to control the PTI mitigation default setting, while Arshan Khanifar benchmarked the performance impact of the in-progress mitigation patches. In addition, Arshan and Mitchell each developed changes to FreeBSDs tool chain to support the full set of mitigations that will be applied.
These mitigations will continue be tested, benchmarked, and refined in FreeBSD-CURRENT before being merged into stable branches and then being made available as updates to FreeBSD releases. Details on the timing of these merges and releases will be shared as they become available.
I would like to acknowledge all of those in the FreeBSD community who have participated in FreeBSDs response to Meltdown and Spectre, for testing, reviewing, and coordinating x86 mitigations, for developing mitigations for other processor architectures and for the Bhyve hypervisor, and for working on the toolchain-based mitigations.
Guides: Getting Started & Lumina Theme Submissions
Guides RepositoryI am pleased to announce the beginning of a new sub-series of blog posts for the Lumina project: Guides!
The TrueOS/Lumina projects want to support our users as they use Lumina or experiment with TrueOS. To that end, weve recently set up a central repository for our users to share instructions or other how-to guides with each other! Project developers and contributors will also submit guides to the repository on occasion, but the overall goal is to provide a simple hub for instructions written by any Lumina or TrueOS user. This will make it easier for users to not only find a how-to for some procedure, but also a very easy way to give back to the community by writing simple instructions or more detailed guides.
Guide: Lumina Themes SubmissionsOur first guide to get the whole thing started was created by the TrueOS Linebacker (with technical assistance from our own q5sys). In this guide, Terry Tate will walk you through the steps necessary to submit new wallpaper images to the Lumina Themes collection. This procedure is fully documented with screenshots every step of the way, walking you through a simple procedure that only requires a web browser and a Github account!
TrueOS Community GuidesThe end result of this guide was that Terry Tate was able to submit this cool new Lunar-4K wallpaper to the lumina-nature collection.
ZFS vs. OpenZFS (by Michael Dexter)
Youve probably heard us say a mix of ZFS and OpenZFS and an explanation is long-overdue. Our Senior Analyst clears up what ZFS and OpenZFS refer to and how they differ.
I admit that we geeks tend to get caught up in the nuts and bolts of enterprise storage and overlook the more obvious questions that users might have. Youve probably noticed that this blog and the FreeNAS blog refer to ZFS and OpenZFS seemingly at random when talking about the amazing file system at the heart of FreeNAS and every storage product that iXsystems sells. I will do my best to clarify what exactly these two terms refer to.
From its inception, ZFS has referred to the Zettabyte File System developed at Sun Microsystems and published under the CDDL Open SWed, 07 Feb 2018 - 1h 35min - 831 - 231: Unix Architecture Evolution
We cover an interview about Unix Architecture Evolution, another vBSDcon trip report, how to teach an old Unix about backspace, new NUMA support coming to FreeBSD, and stack pointer checking in OpenBSD.
This episode was brought to you by
Headlines
Unix Architecture Evolution from the 1970 PDP-7 to the 2017 FreeBSD
Q: Could you briefly introduce yourself?
Q: What will your talk be about, exactly?Im a professor of software engineering, a programmer at heart, and a technology author. Currently Im also the editor in chief of the IEEE Software magazine. I recently published the book Effective Debugging, where I detail 66 ways to debug software and systems.
Q: Why this topic?I will describe how the architecture of the Unix operating system evolved over the past half century, starting from an unnamed system written in PDP-7 assembly language and ending with a modern FreeBSD system. My talk is based, first, on a GitHub repository where I tried to record the systems history from 1970 until today and, second, on the evolution of documented facilities (user commands, system calls, library functions) across revisions. I will thus present the early systems defining architectural features (layering, system calls, devices as files, an interpreter, and process management) and the important ones that followed in subsequent releases: the tree directory structure, user contributed code, I/O redirection, the shell as a user program, groups, pipes, scripting, and little languages.
Q: What do you hope to accomplish by giving this talk? What do you expect?Unix stands out as a major engineering breakthrough due to its exemplary design, its numerous technical contributions, its impact, its development model, and its widespread use. Furthermore, the design of the Unix programming environment has been characterized as one offering unusual simplicity, power, and elegance. Consequently, there are many lessons that we can learn by studying the evolution of the Unix architecture, which we can apply to the design of new systems. I often see modern systems that suffer from a bloat of architectural features and a lack of clear form on which functionality can be built. I believe that many of the modern Unix architecture defining features are excellent examples of what we should strive toward as system architects.
Q: What are the most significant milestones in the development of Unix?Id like FOSDEM attendees to leave the talk with their mind full with architectural features of timeless quality. I want them to realize that architectural elegance isnt derived by piling design patterns and does not need to be expensive in terms of resources. Rather, beautiful architecture can be achieved on an extremely modest scale. Furthermore, I want attendees to appreciate the importance of adopting flexible conventions rather than rigid enforcement mechanisms. Finally, I want to demonstrate through examples that the open source culture was part of Unix from its earliest days.
Q: In which areas has the development of Unix stalled?The architectural development of Unix follows a path of continuous evolution, albeit at a slowing pace, so I dont see here the most important milestones. I would however define as significant milestones two key changes in the way Unix was developed. The first occurred in the late 1970s when significant activity shifted from a closely-knit team of researchers at the AT&T Bell Labs to the Computer Science Research Group in the University of California at Berkeley. This opened the system to academic contributions and growth through competitive research funding. The second took place in the late 1980s and the 1990s when Berkeley open-sourced the the code it had developed (by that time a large percentage of the system) and enthusiasts built on it to create complete open source operating system distributions: 386BSD, and then FreeBSD, NetBSD, OpenBSD, and others.
Q: What are the core features that still link the 1970 PDP-7 system to the latest FreeBSD 11.1 release, almost half a century apart?The data I will show demonstrate that there were in the past some long periods where the number of C library functions and system calls remained mostly stable. Nowadays there is significant growth in the number of all documented facilities with the exception of file formats. Im looking forward to a discussion regarding the meaning of these growth patterns in the Q&A session after the talk.
Q: Can we still add innovative changes to operating systems like FreeBSD without breaking the Unix philosophy? Will there be a moment where FreeBSD isnt recognizable anymore as a descendant of the 1970 PDP-7 system?Over the past half-century the Unix system has grown by four orders of magnitude from a few thousand lines of code to many millions. Nevertheless, looking at a 1970s architecture diagram and a current one reveals that the initial architectural blocks are still with us today. Furthermore, most system calls, user programs, and C library functions of that era have survived until today with essentially similar functionality. Ive even found in modern FreeBSD some lines of code that have survived unchanged for 40 years.
Q: Have you enjoyed previous FOSDEM editions?Theres a saying that form liberates. So having available a time-tested form for developing operating system functionality allows you to innovate in areas that matter rather than reinventing the wheel.
Such concepts include having commands act as a filter, providing manual pages with a consistent structure, supplying build information in the form of a Makefile, installing files in a well-defined directory hierarchy, implementing filesystems with an standardized object-oriented interface, and packaging reusable functions as a library. Within this framework theres ample space for both incremental additions (think of jq, the JSON query command) and radical innovations (consider the Solaris-derived ZFS and dtrace functionality). For this reason I think that BSD and Linux systems will always be recognizable as direct or intellectual descendants of the 1970s Research Unix editions.
Immensely! As an academic I need to attend many scientific conferences and meetings in order to present research results and interact with colleagues. This means too much time spent traveling and away from home, and a limited number of conferences Im in the end able to attend. Nevertheless, attending FOSDEM is an easy decision due to the world-changing nature of its theme, the breadth of the topics presented, the participants enthusiasm and energy, as well as the exemplary, very efficient conference organization.
Another vBSDCon trip report we just found
We just got tipped about another trip report from vBSDCon, this time from one of the first time speakers: W. Dean Freeman
A BSD conference is a very different experience than anything else out there. You have to try it, it is the only way to truly understand it.Recently I had the honor of co-presenting on the internals of FreeBSDs Kernel RNG with John-Mark Gurney at the 3rd biennial vBSDCon, hosted in Reston, VA hosted by Verisign.
Ive been in and out of the FreeBSD community for about 20 years. As Ive mentioned on here before, my first Unix encounter was FreeBSD 2.2.8 when I was in the 7th or 8th grade. However, for all that time Ive never managed to get out to any of the cons. Ive been to one or two BUG meetings and Ive met some folks from IRC before, but nothing like this.
See, he didnt say terrifying. It went very well. You should totally submit a talk for the next conference, even if it is your first.Id also not had to do a stand-up presentation really since college before this. So, my first BSD con and my first time presenting rolled into one made for an interesting experience.
There is nothing that can match the high bandwidth interchange of ideas in person. The internet has made all kinds of communication possible, and we use it all the time, but every once in a while, getting together in person is hugely valuable. Dean then went on to list some of the talks he found most valuable, including DTrace, Capsicum, bhyve, *BSD security tools, and Paul Vixies talk about gets()That said, it was amazing and invigorating experience. I got to meet a few big names in the FreeBSD community, discuss projects, ideas for FreeBSD, etc. I did seem to spend an unusual amount of time talking about FIPS and Common Criteria with folks, but to me thats a good sign and indicative that there is interest in working to close gaps between FreeBSD and the current requirements so that we can start getting FreeBSD and more BSD-based products into the government and start whittling away the domination of Linux (especially since Oracle has cut Solaris, SPARC and the ZFS storage appliance business units).
Then Dean covered some thoughts on his own talk:I think the talk that really had the biggest impact on me, however, was Kyle Kneisls talk on BSD community dynamics. One of the key points he asked was whether the things that drew us to the BSD community in the first place would be able to happen today. Obviously, Im not a 12 or 13 year old kid anymore, but it really got me thinking. That, combined with getting face time with people Id previously only known as screen names has recently drawn me back into participating in IRC and rejoining mailing lists (wdf on freenode. be on the lookout!)
I remember my first solo talk, and GNN asking the right question in the middle to get me to explain a part of it I had missed. It was very helpful.JMG and my talk seems to have been well received, with people paying lots of attention. I dont know what a typical number of questions is for one of these things, but on day one there werent that many questions. We got about 5 during our question time and spent most of the rest of the day fielding questions from interested attendees. Getting a great talk! from GNN after coming down from the stage was probably one of the major highlights for me.
Final Thoughts:I think key to the interest in our presentation was that JMG did a good job framing a very complicated topics importance in terms everyone could understand. It also helped that we got to drop some serious truth bombs.
I met a lot of folks in person for the first time, and met some people Id never known online before. It was a great community and Im glad I got a chance to expand my network.
Verisign were excellent hosts and they took good care of both speakers (covering airfare, rooms, etc.) and also conference attendees at large. The dinners that they hosted were quite good as well.
Im definitely interested in attending vBSDCon again and now that Ive had a taste of meeting IRL with the community on scale of more than a handful, I have every intention of finally making it to BSDCan next year (Id said it in 2017, but then moved to Texas for a new job and it wasnt going to be practical). This year for sure, though!Teaching an Almost 40-year Old UNIX about Backspace
Introduction
Getting the DiffI have been messing with the UNIX® operating system, Seventh Edition (commonly known as UNIX V7 or just V7) for a while now. V7 dates from 1979, so its about 40 years old at this point. The last post was on V7/x86, but since Ive run into various issues with it, I moved on to a proper installation of V7 on SIMH. The Internet has some really good resources on installing V7 in SIMH. Thus, I set out on my own journey on installing and using V7 a while ago, but that was remarkably uneventful.
One convenience that I have been dearly missing since the switch from V7/x86 is a functioning backspace key. There seem to be multiple different definitions of backspace:
BS, as in ASCII character 8 (010, 0x08, also represented as H), and
DEL, as in ASCII character 127 (0177, 0x7F, also represented as ?).
V7 does not accept either for input by default. Instead, # is used as the erase character and @ is used as the kill character. These defaults have been there since UNIX V1. In fact, they have been there since Multics, where they got chosen seemingly arbitrarily. The erase character erases the character before it. The kill character kills (deletes) the whole line. For example, ba##gooo#d would be interpreted as good and bad line@good line would be interpreted as good line.There is some debate on whether BS or DEL is the correct character for terminals to send when the user presses the backspace key. However, most programs have settled on DEL today. tmux forces DEL, even if the terminal emulator sends BS, so simply changing my terminal to send BS was not an option. The change from the defaults outlined here to todays modern-day defaults occurred between 4.1BSD and 4.2BSD. enf on Hacker News has written a nice overview of the various conventions
Closing ThoughtsFor future generations as well as myself when I inevitably majorly break this installation of V7, I wanted to make a diff. However, my V7 is installed in SIMH. I am not a very intelligent man, I didnt keep backup copies of the files Id changed. Getting data out of this emulated machine is an exercise in frustration.
In the end, I printed everything on screen using cat(1) and copied that out. Then I performed a manual diff against the original source code tree because tabs got converted to spaces in the process. Then I applied the changes to clean copies that did have the tabs. And finally, I actually invoked diff(1).
Figuring all this out took me a few days. Penetrating how the system is put together was surprisingly fairly hard at first, but then the difficulty curve eased up. It was an interesting exercise in some kind of reverse engineering and I definitely learned something about tty handling. I was, however, not pleased with using ed(1), even if I do know the basics. vi(1) is a blessing that I did not appreciate enough until recently. Had I also been unable to access recursive grep(1) on my host and scroll through the code, I wouldve probably given up. Writing UNIX under those kinds of editing conditions is an amazing feat. I have nothing but the greatest respect for software developers of those days.
News Roundup
New NUMA support coming to FreeBSD CURRENT
Hello folks,
I am working on merging improved NUMA support with policy implemented by cpuset(2) over the next week. This work has been supported by Dell/EMC's Isilon product division and Netflix. You can see some discussion of these changes here:
https://reviews.freebsd.org/D13403
https://reviews.freebsd.org/D13289
https://reviews.freebsd.org/D13545The work has been done in user/jeff/numa if you want to look at svn history or experiment with the branch. It has been tested by Peter Holm on i386 and amd64 and it has been verified to work on arm at various points.
We are working towards compatibility with libnuma and linux mbind. These commits will bring in improved support for NUMA in the kernel. There are new domain specific allocation functions available to kernel for UMA, malloc, kmem_, and vm_page*. busdmamem consumers will automatically be placed in the correct domain, bringing automatic improvements to some
device performance.cpuset will be able to constrains processes, groups of processes, jails, etc. to subsets of the system memory domains, just as it can with sets of cpus. It can set default policy for any of the above. Threads can use cpusets to set policy that specifies a subset of their visible domains.
Available policies are first-touch (local in linux terms), round-robin (similar to linux interleave), and preferred. For now, the default is round-robin. You can achieve a fixed domain policy by using round-robin with a bitmask of a single domain. As the scheduler and VM become more
sophisticated we may switch the default to first-touch as linux does.Currently these features are enabled with VM_NUMA_ALLOC and MAXMEMDOM. It will eventually be NUMA/MAXMEMDOM to match SMP/MAXCPU. The current NUMA syscalls and VM_NUMA_ALLOC code was 'experimental' and will be deprecated. numactl will continue to be supported although cpuset should be preferred going forward as it supports the full feature set of the new API.
Thank you for your patience as I deal with the inevitable fallout of such sweeping changes. If you do have bugs, please file them in bugzilla, or reach out to me directly. I don't always have time to catch up on all of my mailing list mail and regretfully things slip through the cracks when
they are not addressed directly to me.Thanks,
JeffStack pointer checking OpenBSD
Stefan (stefan@) and I have been working for a few months on this diff, with help from a few others.
At every trap and system call, it checks if the stack-pointer is on a page that is marked MAP_STACK. execve() is changed to create such mappings for the process stack. Also, libpthread is taught the new MAP_STACK flag to use with mmap().
There is no corresponding system call which can set MAP_FLAG on an existing page, you can only set the flag by mapping new memory into place. That is a piece of the security model.
The purpose of this change is to twart stack pivots, which apparently have gained some popularity in JIT ROP attacks. It makes it difficult to place the ROP stack in regular data memory, and then perform a system call from it. Workarounds are cumbersome, increasing the need for far more gadgetry. But also the trap case -- if any memory experiences a demand page fault, the same check will occur and potentially also kill the process.
We have experimented a little with performing this check during device interrupts, but there are some locking concerns and performance may then become a concern. It'll be best to gain experience from handle of syncronous trap cases first.
chrome and other applications I use run fine!
I'm asking for some feedback to discover what ports this breaks, we'd like to know. Those would be ports which try to (unconventionally) create their stacks in malloc()'d memory or inside another Data structure. Most of them are probably easily fixed ...
Qt 5.9 on FreeBSD
Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a proper ports build.
The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way.
Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so its more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speaking of ugh, Im still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux.
For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled a
Thu, 01 Feb 2018 - 1h 24min - 830 - 230: Your questions, Part III
We provide you with updates to Spectre and Meltdown from various BSD projects, a review of TrueOS from Linux, how to set up FreeBSD on ThinkPad x240, and a whole bunch of beastie bits.
This episode was brought to you by
Headlines
KPTI patch lands in FreeBSD -current
After a heroic effort by Konstantin Belousov kib@FreeBSD.org, the first meltdown patch has landed in FreeBSD This creates separate page tables for the Kernel and userland, and switches between them when executions enters the kernel, and when it returns to userland It is currently off by default, but you are encouraged to test it, so it can be merged back to the release branches. Set vm.pmap.pti=1 in /boot/loader.conf The existing implementation of PCID (process-context identifiers), is not compatible with the new PTI code, and is disabled when PTI is enabled, decreasing performance. A future patch will use PCID in a way that is compatible with PTI. PCID allows the OS to annotate memory mappings to specific processes, so that they can be flushed selectively, and so that they are only used when in the context of that application. Once the developers are relatively confident in the correctness of the code that has landed in -current, it will be ported back to FreeBSD 10 and 11, and released as a security advisory. Apparently porting back to FreeBSD 11 only has some relatively simple merge conflicts, but 10 will be more work. Former FreeBSD Security Officer Dag-Erling Smørgrav has created a meltdown testing and PoC tool that you can use to check your system. It is not finished yet, and doesnt seem to work with newer processors (haswell and newer). The first partial mitigation for Spectre variant 2 for bhyve on AMD64 has also been committed The latest information is always available on the FreeBSD Wiki ***Some thoughts on Spectre and Meltdown
Colin Percival breaks down how these vulnerabilities work, with same nice analogies What is a side channel:
Spectre Variant 1:I want to know when my girlfriend's passport expires, but she won't show me her passport (she complains that it has a horrible photo) and refuses to tell me the expiry date. I tell her that I'm going to take her to Europe on vacation in August and watch what happens: If she runs out to renew her passport, I know that it will expire before August; while if she doesn't get her passport renewed, I know that it will remain valid beyond that date. Her desire to ensure that her passport would be valid inadvertently revealed to me some information: Whether its expiry date was before or after August.
Spectre Variant 2:I tell my girlfriend that I'm going to take her on vacation in June, but I don't tell her where yet; however, she knows that it will either be somewhere within Canada (for which she doesn't need a passport, since we live in Vancouver) or somewhere in Europe. She knows that it takes time to get a passport renewed, so she checks her passport and (if it was about to expire) gets it renewed just in case I later reveal that I'm going to take her to Europe. If I tell her later that I'm only taking her to Ottawa well, she didn't need to renew her passport after all, but in the meantime her behaviour has already revealed to me whether her passport was about to expire. This is what Google refers to "variant 1" of the Spectre vulnerability: Even though she didn't need her passport, she made sure it was still valid just in case she was going to need it.
Meltdown:I spend a week talking about how Oxford is a wonderful place to visit and I really enjoyed the years I spent there, and then I tell her that I want to take her on vacation. She very reasonably assumes that since I've been talking about Oxford so much I must be planning on taking her to England, and runs off to check her passport and potentially renew it... but in fact I tricked her and I'm only planning on taking her to Ottawa.
I tell my girlfriend that I want to take her to the Korean peninsula. She knows that her passport is valid for long enough; but she immediately runs off to check that her North Korean visa hasn't expired. Why does she have a North Korean visa, you ask? Good question. She doesn't but she runs off to check its expiry date anyway! Because she doesn't have a North Korean visa, she (somehow) checks the expiry date on someone else's North Korean visa, and then (if it is about to expire) runs out to renew it and so by telling her that I want to take her to Korea for a vacation I find out something she couldn't have told me even if she wanted to.
Final thoughts on vulnerability disclosure
The way these issues were handled was a mess; frankly, I expected better of Google, I expected better of Intel, and I expected better of the Linux community. When I found that Hyper-Threading was easily exploitable, I spent five months notifying the security community and preparing everyone for my announcement of the vulnerability; but when the embargo ended at midnight UTC and FreeBSD published its advisory a few minutes later, the broader world was taken entirely by surprise. Nobody knew what was coming aside from the people who needed to know; and the people who needed to know had months of warning.
Contrast that with what happened this time around. Google discovered a problem and reported it to Intel, AMD, and ARM on June 1st. Did they then go around contacting all of the operating systems which would need to work on fixes for this? Not even close. FreeBSD was notified the week before Christmas, over six months after the vulnerabilities were discovered. Now, FreeBSD can occasionally respond very quickly to security vulnerabilities, even when they arise at inconvenient times on November 30th 2009 a vulnerability was reported at 22:12 UTC, and on December 1st I provided a patch at 01:20 UTC, barely over 3 hours later but that was an extremely simple bug which needed only a few lines of code to fix; the Spectre and Meltdown issues are orders of magnitude more complex.
To make things worse, the Linux community was notified and couldn't keep their mouths shut. Standard practice for multi-vendor advisories like this is that an embargo date is set, and nobody does anything publicly prior to that date. People don't publish advisories; they don't commit patches into their public source code repositories; and they definitely don't engage in arguments on public mailing lists about whether the patches are needed for different CPUs. As a result, despite an embargo date being set for January 9th, by January 4th anyone who cared knew about the issues and there was code being passed around on Twitter for exploiting them.
This is not the first time I've seen people get sloppy with embargoes recently, but it's by far the worst case. As an industry we pride ourselves on the concept of responsible disclosure ensuring that people are notified in time to prepare fixes before an issue is disclosed publicly but in this case there was far too much disclosure and nowhere near enough responsibility. We can do better, and I sincerely hope that next time we do.CPU microcode update code for amd64
Patrick Wildt (patrick@) recently committed some code that will update the Intel microcode on many Intel CPUs, a diff initially written by Stefan Fritsch (sf@). The microcode of your CPU is basically the firmware that runs on your (Intel) processor, defining its instruction set in terms of so called "microinstructions". The new code depends, of course, on the corresponding firmware package, ported by Patrick which can be installed using a very recent fw_update(1). Of course, this all plays into the recently revealed problems in Intel (and other) CPUs, Meltdown and Spectre. Now Theo has explained the workings of the code on openbsd-tech, detailing some of the challenges in updating microcode on CPUs where your OS is already starting to run. Theo hints at future updates to the intel-firmware package in his mail:
Several developers are working on mitigations for these issues, attacking the problem from several angles. Expect to see more updates to a CVS tree near you soon.Patrick and others committed amd64 Intel cpu microcode update code over the last few days. The approach isn't perfect, but it is good enough for a start. I want to explain the situation.
When you fw_update, you'll get the firmware files.
Upon a reboot, it will attempt to update the microcode on your cpus.
Maybe there isn't a new microcode. Maybe your BIOS has a copy of the microcode and installs it before booting OpenBSD.
This firmware installation is done a little late. Doing it better will require some work in the bootblocks to find the firmware files, but time is a bit short to do that right now.
The branch-target-cache flushing features added in new microcode are not being used yet. There is more code which has to be written, but again other work is happening first.
Also, Intel is saying their new microcodes sucks and people should wait a little.
"Hi, my name is Intel and I'm an cheating speculator".Intel: as a *BSD user, I am fucking pissed!
Meltdown, Spectre and BSD the pissed partI wasnt going to write anything on the recently found x64 architecture related bugs. Im not a kernel developer nor even a programmer and I cant say that I have a solid understanding of what Meltdown and Spectre attacks are. Also there already is a ton of articles and posts written by people who have no grasp of the subject.
Im however a malcontent and I find this a good way to express my feelings:Intel: as a *BSD user, I am fucking pissed!
BSD user base too small?Part of my work is UNIX-like systems administration including BSDs and Linuces. As much as I am happy with Linux changes already made, I am beyond pissed about how the BSDs were handled by Intel because they were not. FreeBSD Security Team received some heads-up just before Xmas, while OpenBSD, NetBSD and DragonflyBSD teams received no prior warnings.
Meltdown and Spectre attacks are hard to perform. It is a hard work to mitigate them in the software, as the bugs lay in the CPUs and are not fixable by microcode updates. Developers are trying to mitigate these bugs in a way that will deliver smallest performance losses. A lot of time consuming work is needed to fix CPU vendors mistakes. Linux developers had this time. BSD developers did not.
BSD user base is small in comparison to Linux. Seems that its too small for Intel. PlayStation4 consoles are FreeBSD-based (and use AMD CPUs) but I think its safe to say that gaming devices are not the most important systems to be fixed. Netflix serves their content off FreeBSD but the bugs are not remotely exploitable (possibly not including JavaScript, but its running someones code locally) so theres probably not much harm to be done here either.
However gamers and Netflix arent the only ones who use *BSD systems. Id say that there is more than a few FreeBSD, NetBSD, OpenBSD and DragonFlyBSD servers on the internet.In March 2017, Intel promised more timely support to FreeBSD. They knew about flaws in their CPUs in June and decided that a timely manner is the end of December short before the embargo was to be lifted.
Intel and Google (probably Intel more): it was your job to pick the correct people to whom the bugs can be disclosed. In my humble opinion you chose poorly by disclosing these issues with ONLY Apple, Microsoft, and the Linux Foundation, of OS vendors. You did much harm to the BSD community.
Intel: Its your bugs. And you offered more support to the FreeBSD Foundation less than 3 months prior to being informed (my guess is that you knew much earlier) on the flaws in YOUR products. I dont want to write more here as the wording would be too strong.
Interview - Viewer Questions
These days, do you consider yourself more of an programmer or a sysadmin? Which one do you enjoy more? Does FreeBSD/BSD enable your business or would another OS suit your needs just as well? Youve hinted that you use FreeBSD as part of your business. Can you elaborate on that and give some technical detail on how its used in that environment? If you were allowed three wishes for anything at all to be implemented or changed in ZFS, what would they be, and why? Per Dataset throughput and IOPS limiting Per-File Cloning and/or zfsmv (move a file from one dataset to another, without copying) Cluster support Allan, you have previously mentioned that you have worked on FreeBSD on MIPS, what made you choose the Onion Omega over something like the Raspberry Pi? What is BSD Nows association with Jupiter broadcasting, and how did the relationship come to be? Jupiter seems to be associated with several Linux-themed podcasts, and Im wondering how and why BSD Now joined Jupiter. The two communities (the Linuxes and BSDs) dont always seem to mix freely -- or do they? What kind of keyboard is that? Have you ever tried an ErgoDox? The ErgoDox EZ is made by a Canadian. You mentioned when doing one of your talks on UCL for FreeBSD that you had only recently learned C. I am also aware of your history also on contributing to the FreeBSD handbook and to documentation in general. Given you started with C relatively recently, what made you want to learn it, how quickly did you pick it up, and is it your favourite language? It is most inspiring to me, as you are clearly so talented, and of all the languages I have learned (including C++), I still prefer C in my heart of hearts. I'd be really interested to hear your answer, many thanks. ***News Roundup
LinuxAndUbuntu Review Of TrueOS A Unix Based OS
Trust me, the name TrueOS takes me back to 1990s when Tru64 UNIX operating system made its presence. TrueOS is PC-BSDs new unified brand built upon FreeBSD-CURRENT code base. Note that TrueOS is not a Linux distro but is BSD Unix. FreeBSD is known for its cutting-edge features, security, scalability, and ability to work both as a server and desktop operating system. TrueOS aims at having user-friendliness with the power of FreeBSD OS. Let us start with going into details of different aspects of the TrueOS.
TrueOS History
?TrueOS was founded by Kris Moore in 2005 with name PC-BSD. Initial version focused to make FreeBSD easy to use starting with providing GUI based installer (to relatively complicated FreeBSD installer). In the year 2006, PC-BSD was acquired by iXsystems. Before rebranding as TrueOS in Sept 2016, PC-BSD reached a stage starting considering better than vanilla FreeBSD. Older PC-BSD version used to support both x86 and x86-64 architecture.
Kris Moore, the developer founder, says about rebranding: Weve already been using TrueOS for the server side of PC-BSD, and it made sense to unify the names. PC-BSD doesnt reflect server or embedded well. TrueOS Desktop/Server/Embedded can be real products, avoids some of the alphabet soup, and gives us a more catchy name.TrueOS First Impression
?The startup is little longer; may be due to starting up of many services. The heavy KDE well suited to PC-BSD. The C++/Qt5 based Lumina desktop environment is light and fast. The Lumina offers an easy way to configure menu and panels. I did not face any problems for continuous use of two weeks on a virtual machine having the minimal configuration: 1 GB RAM, 20 GB hard disk and Intel 3.06 GHz i3 processor. The Lumina desktop is light and fast. The developers of Lumina know what they are doing and have a good idea of what makes a good IDE. As it happens with any new desktop environment, it needs some time to settle. Let us hope that they keep to the path they are on with it.
Conclusion
?The TrueOS is impressive when consider it as relatively young. It is a daring step that TrueOS developers took FreeBSD Current rather than FreeBSD Stable code base. Overall it has created its own place from the legacy shadow of PC-BSD. Starting with easy installation TrueOS is a good combination of software and utilities that make the system ready to use. Go and get a TrueOS ISO to unleash the bleeding edge tag of FreeBSD
Thinkpad x240 - FreeBSD Setup
What follows is a record of how I set up FreeBSD to be my daily driver OS on the Lenovo Thinkpad X240. Everything seems to work great. Although, the touchpad needs some tweaking. I've tried several configurations, even recompiling Xorg with EVDEV support and all that, to no avail. Eventually I will figure it out. Do not sleep the laptop from the command line. Do it from within Xorg, or it will not wake up. I don't know why. You can do it from a terminal within Xorg, just not from the naked command line without Xorg started. It also will not sleep by closing the lid. I included a sudo config that allows you to run /usr/sbin/zzz without a password, so what I do is I have a key combo assigned within i3wm to run "sudo /usr/sbin/zzz". It works fine this way.
I go into detail when it comes to setting up Xorg with i3wm. You can skip this if you want, but if you've never used a tiling window manager, it will handle screen real estate very efficiently on a laptop with a 12.5-inch screen and a touchpad.
First, download the amd64 image for 11.1-RELEASE and flash it to a USB pen drive. For the Unices, use this:
# dd if=FreeBSD-11.1-RELEASE-amd64-memstick.img of=/dev/da0 bs=1M conv=syncObviously, you'll change /dev/da0 to whatever the USB pen drive is assigned. Plug it in, check dmesg.
Leave it plugged in, restart the laptop. When prompted, tap Enter to halt the boot process, then F12 to select a bootable device. Choose the USB drive.
I won't go through the actual install process, but it is pretty damn easy so just look at a guide or two and you'll be fine. If you can install Debian, you can install FreeBSD. I will, however, recommend ZFS if you have over 4GB of RAM (my particular variant of the X240 has 8GB of RAM, so yours should have at least 4GB), along with an encrypted disk, and an encrypted SWAP partition. When prompted to add an additional user, and you get to the question where it asks for additional groups, please make sure you add the user to "wheel". The rest should be self-explanatory during the install.
Now for the good shit. You just booted into a fresh FreeBSD install. Now what? Well, time to fire up vi and open some config files...
CNN Article about CDROM.com and FreeBSD, from 1999
The hardwareWalnut Creek CDROM sells a lot of CD-ROMs, but it gives away even more data. Specifically, anyone who has Internet access is free to log into wcarchive (ftp.cdrom.com) and start downloading bits.
Even with a good Internet connection, however, you should expect to be at it for a while. At the present time, wcarchive resides on half a terabyte (500 GB) of RAID 5-disk storage. Even if your 56-Kbps modem can deliver seven kilobytes per second, downloading the complete archive would take you 70 million seconds. Even then, some of the files would be more than two years out of date, so a bit of "back and fill" would be needed.
Of course, nobody uses wcarchive that way. Instead, they just drop in when they need the odd file or two. The FTP server is very accommodating; 3,600 simultaneous download sessions is the current limit and an upgrade to 10,000 sessions is in the works.
This translates to about 800 GB per day of downloads. Bob Bruce (Walnut Creek's founder) says he's thinking about issuing a press release when they reach a terabyte a day. But 800 GB isn't all that shabby....Because FTP archives don't do a lot of thinking, wcarchive doesn't need a massive cluster of CPUs. In fact, it gets by with a single 200-MHz P6 Pentium Pro and a measly(!) 1 GB of RAM. The I/O support, however, is fairly impressive.
A six-channel Mylex RAID controller (DAC960SXI; Ultra-Wide SCSI-SCSI) is the centerpiece of the I/O subsystem. Two channels link it to the PC ("Personal Computer"!?!), via a dual-channel Adaptec card (AHA-3940AUW; PCI to Ultra-Wide SCSI). An 256-MB internaWed, 24 Jan 2018 - 1h 56min - 829 - 229: The Meltdown of Spectre
We review Meltdown and Spectre responses from various BSD projects, show you how to run CentOS with bhyve, GhostBSD 11.1 is out, and we look at the case against the fork syscall.
This episode was brought to you by
Headlines
More Meltdown
Much has been happened this week, but before we get into a status update of the various mitigations on the other BSDs, some important updates:
Intel has recalled the microcode update they issued on January 8th. It turns out this update can cause Haswell and Broadwell based systems to randomly reboot, with some frequency. AMD has confirmed that its processors are vulnerable to both variants of Spectre, and the the fix for variant #2 will require a forthcoming microcode update, in addition to OS level mitigations Fujitsu has provided a status report for most of its products, including SPARC hardwareThe Register of course has some commentary
Response from OpenBSD:If new code is needed, Intel will need to get it right: the company already faces numerous class action lawsuits. Data centre operators already scrambling to conduct unplanned maintenance will not be happy about the fix reducing stability.
AMD has said that operating system patches alone will address the Spectre bounds check bypass bug. Fixing Spectres branch target injection flaw will require firmware fixes that AMD has said will start to arrive for Ryzen and EPYC CPUs this week.
The Register has also asked other server vendors how theyre addressing the bugs. Oracle has patched its Linux, but has told us it has No comment/statement on this as of now in response to our query about its x86 systems, x86 cloud, Linux and Solaris on x86. The no comment regarding Linux is odd as fixes for Oracle Linux landed here on January 9th.
SPARC-using Fujitsu, meanwhile, has published advice (PDF) revealing how it will address the twin bugs in its servers and PCs, and also saying its SPARC systems are under investigation.'Meltdown, aka "Dear Intel, you suck"'
Theo de Raadt's response to Meltdown
That time in 2007 when Theo talked about how Intel x86 had major design problems in their chips
OpenBSD gets a Microcode updater
Response from Dragonfly BSD:
One Two Three Four
The longer response in four commitsDragonflyBSD master now has full IBRS and IBPB support
IBRS (Indirect Branch Restricted Speculation): The x86 IBRS feature requires corresponding microcode support. It mitigates the variant 2 vulnerability. If IBRS is set, near returns and near indirect jumps/calls will not allow their predicted target address to be controlled by code that executed in a less privileged prediction mode before the IBRS mode was last written with a value of 1 or on another logical processor so long as all RSB entries from the previous less privileged prediction mode are overwritten. Speculation on Skylake and later requires these patches ("dynamic IBRS") be used instead of retpoline. If you are very paranoid or you run on a CPU where IBRS=1 is cheaper, you may also want to run in "IBRS always" mode. IBPB (Indirect Branch Prediction Barrier): Setting of IBPB ensures that earlier code's behavior does not control later indirect branch predictions. It is used when context switching to new untrusted address space. Unlike IBRS, IBPB is a command MSR and does not retain its state.DragonFlyBSD's Meltdown Fix Causing More Slowdowns Than Linux
NetBSD SVS (Separate Virtual Space)
Running CentOS with Bhyve
(zroot in this case is the zpool I am using)With the addition of UEFI in FreeBSD (since version 11), users of bhyve can use the UEFI boot loader instead of the grub2-bhyve port for booting operating systems such as Microsoft Windows, Linux and OpenBSD. The following page provides information necessary for setting up bhyve with UEFI boot loader support:
https://wiki.freebsd.org/bhyve/UEFI
Features have been added to vmrun.sh to make it easier to setup the UEFI boot loader, but the following is required to install the UEFI firmware pkg:# pkg install -y uefi-edk2-bhyveWith graphical support, you can use a vnc client like tigervnc, which can be installed with the following command:
# pkg install -y tigervncIn the case of most corporate or government environments, the Linux of choice is RHEL, or CentOS. Utilizing bhyve, you can test and install CentOS in a bhyve VM the same way you would deploy a Linux VM in production. The first step is to download the CentOS iso (for this tutorial I used the CentOS minimal ISO): http://isoredirect.centos.org/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1708.iso
I normally use a ZFS Volume (zvol) when running bhyve VMs. Run the following commands to create a zvol (ensure you have enough disk space to perform these operations):
# zfs create -V20G -o volmode=dev zroot/centos0
(replace em0 with whatever your physical interface is).Similar to my previous post about vmrun.sh, you need certain items to be configured on FreeBSD in order to use bhyve. The following commands are necessary to get things running:
# echo "vfs.zfs.vol.mode=2" >> /boot/loader.conf # kldload vmm # ifconfig tap0 create # sysctl net.link.tap.up_on_open=1 net.link.tap.up_on_open: 0 -> 1 # ifconfig bridge0 create # ifconfig bridge0 addm em0 addm tap0 # ifconfig bridge0 upThere are a number of utilities that can be used to manage bhyve VMs, and I am sure there is a way to use vmrun.sh to run Linux VMs, but since all of the HowTos for running Linux use the bhyve command line, the following script is what I use for running CentOS with bhyve.
#!/bin/sh # # General bhyve install/run script for CentOS # Based on scripts from pr1ntf and lattera HOST="127.0.0.1" PORT="5901" ISO="/tmp/centos.iso" VMNAME="centos" ZVOL="centos0" SERIAL="nmda0A" TAP="tap1" CPU="1" RAM="1024M" HEIGHT="800" WIDTH="600" if [ "$1" == "install" ]; then #Kill it before starting it bhyvectl --destroy --vm=$VMNAME bhyve -c $CPU -m $RAM -H -P -A \ -s 0,hostbridge \ -s 2,virtio-net,$TAP \ -s 3,ahci-cd,$ISO \ -s 4,virtio-blk,/dev/zvol/zroot/$ZVOL \ -s 29,fbuf,tcp=$HOST:$PORT,w=$WIDTH,h=$HEIGHT \ -s 30,xhci,tablet \ -s 31,lpc -l com1,/dev/$SERIAL \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ $VMNAME #kill it after bhyvectl --destroy --vm=$VMNAME elif [ "$1" == "run" ]; then #Kill it before starting it bhyvectl --destroy --vm=centos bhyve -c $CPU -m $RAM -w -H \ -s 0,hostbridge \ -s 2,virtio-net,$TAP \ -s 4,virtio-blk,/dev/zvol/zroot/$ZVOL \ -s 29,fbuf,tcp=$HOST:$PORT,w=$WIDTH,h=$HEIGHT \ -s 30,xhci,tablet \ -s 31,lpc -l com1,/dev/$SERIAL \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ $VMNAME & else echo "Please type install or run"; fiThe variables at the top of the script can be adjusted to fit your own needs. With the addition of the graphics output protocol in UEFI (or UEFI-GOP), a VNC console is launched and hosted with the HOST and PORT setting. There is a password option available for the VNC service, but the connection should be treated as insecure. It is advised to only listen on localhost with the VNC console and tunnel into the host of the bhyve VM. Now with the ISO copied to /tmp/centos.iso, and the script saved as centos.sh you can run the following command to start the install:
# ./centos.sh installAt this point, using vncviewer (on the local machine, or over an SSH tunnel), you should be able to bring up the console and run the CentOS installer as normal. The absolutely most critical item is to resolve an issue with the booting of UEFI after the installation has completed. Because of the path used in bhyve, you need to run the following to be able to boot CentOS after the installation:
# cp -f /mnt/sysimage/boot/efi/EFI/centos/grubx64.efi /mnt/sysimage/boot/efi/EFI/BOOTWith this setting changed, the same script can be used to launch your CentOS VM as needed:
# ./centos.sh runIf you are interested in a better solution for managing your Linux VM, take a look at the various bhyve management ports in the FreeBSD ports tree.
Interview - newnix architect - @newnix
News Roundup
GhostBSD 11.1 - FreeBSD for the desktop
InstallingGhostBSD is a desktop oriented operating system which is based on FreeBSD. The project takes the FreeBSD operating system and adds a desktop environment, some popular applications, a graphical package manager and Linux binary compatibility. GhostBSD is available in two flavours, MATE and Xfce, and is currently available for 64-bit x86 computers exclusively. I downloaded the MATE edition which is available as a 2.3GB ISO file.
Early impressionsGhostBSD's system installer is a graphical application which begins by asking us for our preferred language, which we can select from a list. We can then select our keyboard's layout and our time zone. When it comes to partitioning we have three main options: let GhostBSD take over the entire disk using UFS as the file system, create a custom UFS layout or take over the entire disk using ZFS as the file system. UFS is a classic file system and quite popular, it is more or less FreeBSD's equivalent to Linux's ext4. ZFS is a more advanced file system with snapshots, multi-disk volumes and optional deduplication of data. I decided to try the ZFS option.
Once I selected ZFS I didn't have many more options to go through. I was given the chance to set the size of my swap space and choose whether to set up ZFS as a plain volume, with a mirrored disk for backup or in a RAID arrangement with multiple disks. I stayed with the plain, single disk arrangement. We are then asked to create a password for the root account and create a username and password for a regular user account. The installer lets us pick our account's shell with the default being fish, which seemed unusual. Other shells, including bash, csh, tcsh, ksh and zsh are available. The installer goes to work copying files and offers to reboot our computer when it is done.
ConclusionsThe newly installed copy of GhostBSD boots to a graphical login screen where we can sign into the account we created during the install process. Signing into our account loads the MATE 1.18 desktop environment. I found MATE to be responsive and applications were quick to open. Early on I noticed odd window behaviour where windows would continue to slide around after I moved them with the mouse, as if the windows were skidding on ice. Turning off compositing in the MATE settings panel corrected this behaviour. I also found the desktop's default font (Montserrat Alternates) to be hard on my eyes as the font is thin and, for lack of a better term, bubbly. Fonts can be easily adjusted in the settings panel.
A few minutes after I signed into my account, a notification appeared in the system tray letting me know software updates were available. Clicking the update icon brings up a small window showing us a list of package updates and, if any are available, updates to the base operating system. FreeBSD, and therefore GhostBSD, both separate the core operating system from the applications (packages) which run on the operating system. This means we can update the core of the system separately from the applications. GhostBSD's core remains relatively static and minimal while applications are updated using a semi-rolling schedule.
When we are updating the core operating system, the update manager will give us the option of rebooting the system to finish the process. We can dismiss this prompt to continue working, but the wording of the prompt may be confusing. When asked if we want to reboot to continue the update process, the options presented to us are "Continue" or "Restart". The Continue option closes the update manager and returns us to the MATE desktop.
The update manager worked well for me and the only issue I ran into was when I dismissed the update manager and then wanted to install updates later. There are two launchers for the update manager, one in MATE's System menu and one in the settings panel. Clicking either of these launchers didn't accomplish anything. Running the update manager from the command line simply caused the process to lock up until killed. I found if I had dismissed the update manager once, I'd have to wait until I logged in again to use it. Alternatively, I could use a command line tool or use the OctoPkg package manager to install package updates.
Most of my time with GhostBSD, I was impressed and happy with the operating system. GhostBSD builds on a solid, stable FreeBSD core. We benefit from FreeBSD's performance and its large collection of open source software packages. The MATE desktop was very responsive in my trial and the system is relatively light on memory, even when run on ZFS which has a reputation for taking up more memory than other file systems.
FreeBSD Looks At Making Wayland Support Available By Default
FreeBSD vote thread TrueOS Fliped the switch alreadyThere's an active discussion this week about making Wayland support available by default on FreeBSD.
FreeBSD has working Wayland support -- well, assuming you have working Intel / Radeon graphics -- and do have Weston and some other Wayland components available via FreeBSD Ports. FreeBSD has offered working Wayland support that is "quite usable" for more than one year. But, it's not too easy to get going with Wayland on FreeBSD.
Right now those FreeBSD desktop users wanting to use/develop with Wayland currently need to rebuild the GTK3 tool-kit, Mesa, and other packages with Wayland support enabled. This call for action now is about allowing the wayland=on to be made the default.
This move would then allow these dependencies to be built with Wayland support by default, but for the foreseeable future FreeBSD will continue defaulting to X.Org-based sessions.
The FreeBSD developers mostly acknowledge that Wayland is the future and the cost of enabling Wayland support by default is just slightly larger packages, but that weight is still leaner than the size of the X.Org code-base and its dependencies.
fork is not my favorite syscall
This article has been on my to-write list for a while now. In my opinion, fork is one of the most questionable design choices of Unix. I dont understand the circumstances that led to its creation, and I grieve over the legacy rationale that keeps it alive to this day.
Lets set the scene. Its 1971 and youre a fly on the wall in Bell Labs, watching the first edition of Unix being designed for the PDP-11/20. This machine has a 16-bit address space with no more than 248 kilobytes of memory. Theyre discussing how theyre going to support programs that spawn new programs, and someone has a brilliant idea. What if we copied the entire address space of the program into a new process running from the same spot, then let them overwrite themselves with the new program? This got a rousing laugh out of everyone present, then they moved on to a better design which would become immortalized in the most popular and influential operating system of all time.
At least, thats the story Id like to have been told. In actual fact, the laughter becomes consensus. Theres an obvious problem with this approach: every time you want to execute a new program, the entire process space is copied and promptly discarded when the new program begins. Usually when I complain about fork, this the point when its supporters play the virtual memory card, pointing out that modern operating systems dont actually have to copy the whole address space. Well get to that, but first First Edition Unix does copy the whole process space, so this excuse wouldnt have held up at the time. By Fourth Edition Unix (the next one for which kernel sources survived), they had wisened up a bit, and started only copying segments when they faulted.
This model leads to a number of problems. One is that the new process inherits all of the parents process descriptors, so you have to close them all before you exec another process. However, unless youre manually keeping tabs on your open file descriptors, there is no way to know what file handles you must close! The hack that solves this is CLOEXEC, the first of many hacks that deal with forks poor design choices. This file descriptors problem balloons a bit - consider for example if you want to set up a pipe. You have to establish a piped pair of file descriptors in the parent, then close every fd but the pipe in the child, then dup2 the pipe file descriptor over the (now recently closed) file descriptor 1. By this point youve probably had to do several non-trivial operations and utilize a handful of variables from the parent process space, which hopefully were on the stack so that we dont end up copying segments into the new process space anyway.
These problems, however, pale in comparison to my number one complaint with the fork model. Fork is the direct cause of the stupidest component Ive ever heard of in an operating system: the out-of-memory (aka OOM) killer. Say you have a process which is using half of the physical memory on your system, and wants to spawn a tiny program. Since fork copies the entire process, you might be inclined to think that this would make fork fail. But, on Linux and many other operating systems since, it does not fail! They agree that its stupid to copy the entire process just to exec something else, but because fork is Important for Backwards Compatibility, they just fake it and reuse the same memory map (except read-only), then trap the faults and actually copy later. The hope is that the child will get on with it and exec before this happens.
However, nothing prevents the child from doing something other than exec - its free to use the memory space however it desires! This approach now leads to memory overcommittment - Linux has promised memory it does not have. As a result, when it really does run out of physical memory, Linux will just kill off processes until it has some memory back. Linux makes an awfully big fuss about never breaking userspace for a kernel that will lie about memory it doesnt have, then kill programs that try to use the back-alley memory they were given. That this nearly 50 year old crappy design choice has come to this astonishes me.
Alas, I cannot rant forever without discussing the alternatives. There are better process models that have been developed since Unix!
The first attempt I know of is BSDs vfork syscall, which is, in a nutshell, the same as fork but with severe limitations on what you do in the child process (i.e. nothing other than calling exec straight away). There are loads of problems with vfork. It only handles the most basic of use cases: you cannot set up a pipe, cannot set up a pty, and cant even close open file descriptors you inherited from the parent. Also, you couldnt really be sure of what variables you were and werent editing or allowed to edit, considering the limitations of the C specification. Overall this syscall ended up being pretty useless.
Another model is posix_spawn, which is a hell of an interface. Its far too complicated for me to detail here, and in my opinion far too complicated to ever consider using in practice. Even if it could be understood by mortals, its a really bad implementation of the spawn paradigm it basically operates like fork backwards, and inherits many of the same flaws. You still have to deal with children inheriting your file descriptors, for example, only now you do it in the parent process. Its also straight-up impossible to make a genuine pipe with posix_spawn. (Note: a reader corrected me - this is indeed possible via posix_spawn_file_actions_adddup2.)
Lets talk about the good models - rfork and spawn (at least, if spawn is done right). rfork originated from plan9 and is a beautiful little coconut of a syscall, much like the rest of plan9. They also implement fork, but its a special case of rfork. plan9 does not disti
Wed, 17 Jan 2018 - 1h 48min - 828 - 228: The Spectre of Meltdown
We review the information about Spectre & Meltdown thus far, we look at NetBSD memory sanitizer progress, Postgres on ZFS & show you a bit about NomadBSD.
This episode was brought to you by
Headlines
Meltdown Spectre
Official Site Kernel-memory-leaking Intel processor design flaw forces Linux, Windows redesign Intel’s official response The Register mocks intels response with pithy annotations Intel’s Analysis PDF XKCD Response from FreeBSD FreeBSD's patch WIP Why Raspberry Pi isn’t vulnerable to Spectre or Meltdown Xen mitigation patches Overview of affected FreeBSD Platforms/Architectures Groff's response ##### We’ll cover OpenBSD, NetBSD, and DragonflyBSD’s responses in next weeks episode. *** ###The LLVM Memory Sanitizer support work in progress > In the past 31 days, I've managed to get the core functionality of MSan to work. This is an uninitialized memory usage detector. MSan is a special sanitizer because it requires knowledge of every entry to the basesystem library and every entry to the kernel through public interfaces. This is mandatory in order to mark memory regions as initialized. Most of the work has been done directly for MSan. However, part of the work helped generic features in compiler-rt. Sanitizers > Changes in the sanitizer are listed below in chronological order. Almost all of the changes mentioned here landed upstream. A few small patches were reverted due to breaking non-NetBSD hosts and are rescheduled for further investigation. I maintain these patches locally and have moved on for now to work on the remaining features. NetBSD syscall hooks > I wrote a large patch (815kb!) adding support for NetBSD syscall hooks for use with sanitizers. NetBSD ioctl(2) hooks > Similar to the syscall hooks, there is need to handle every ioctl(2) call. I've created the needed patch, this time shorter - for less than 300kb. New patches still pending for upstream review > There are two corrections that I've created, and they are still pending upstream for review: Add MSan interceptor for fstat(2)](https://reviews.llvm.org/D41637) Correct the setitimer interceptor on NetBSD)](https://reviews.llvm.org/D41502) > I've got a few more local patches that require cleanup before submitting to review. NetBSD basesystem corrections Sanitizers in Go The MSan state as of today Solaris support in sanitizers > I've helped the Solaris team add basic support for Sanitizers (ASan, UBsan). This does not help NetBSD directly, however indirectly it improves the overall support for non-Linux hosts and helps to catch more Linuxisms in the code. Plan for the next milestone > I plan to continue the work on MSan and correct sanitizing of the NetBSD basesystem utilities. This mandates me to iterate over the basesystem libraries implementing the missing interceptors and correcting the current support of the existing ones. My milestone is to build all src/bin programs against Memory Sanitizer and when possible execute them cleanly. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate *** ##News Roundup ###MWL’s 2017 Wrap-Up > The obvious place to start is my 2016 wrap-up post](https://blather.michaelwlucas.com/archives/2822), where I listed goals for 2017. As usual, these goals were wildly delusional. > The short answer is, my iron was back up to normal. My writing speed wasn’t, though. I’d lost too much general health, and needed hard exercise to recover it. Yes, writing requires physical endurance. Maintaining that level of concentration for several hours a day demands a certain level of blood flow to the brain. I could have faked it in a day job, but when self-employed as an artist? Not so much. > Then there’s travel. I did my usual BSDCan trip, plus two educational trips to Lincoln City, Oregon. The current political mayhem convinced me that if I wanted to hit EuroBSDCon any time in the next few years, I should do it in the very near future. So I went to Paris, where I promptly got pickpocketed. (Thankfully, they didn’t get my passport.) I was actively writing the third edition of Absolute FreeBSD, so I visited BSDCam in Cambridge to get the latest information and a sense of where FreeBSD was going. I also did weekends at Kansas LinuxFest (because they asked and paid for my trip) and Penguicon. > (Because people will ask: why EuroBSDCon and not AsiaBSDCon? A six-hour transatlantic flight requires that I take a substantial dose of heavy-grade tranquilizers. I’m incapable of making intelligent decisions while on those drugs, or for several hours afterward. They don’t last long enough for twelve-hour flight to Japan, so I need to be accompanied by someone qualified to tell me when I need to take the next dose partway through the flight. This isn’t a predetermined time that I can set an alarm for; it depends on how the clonazepam affects me at those altitudes. A drug overdose while flying over the North Pole would be bad. When I can arrange that qualified companion, I’ll make the trip.) > I need most of the preceding week to prepare for long trips. I need the following week to recover from time shifts and general exhaustion. Additionally, I have to hoard people juice for a few weeks beforehand so I can deal with folks during these expeditions. Travel disrupts my dojo time as well, which impacts my health. > Taken as a whole: I didn’t get nearly as much done as I hoped. I wrote more stories, but Kris Rusch bludgeoned me into submitting them to trad markets. (The woman is a brute, I tell you. Cross her at your peril.) Among my 2017 titles, my fiction outsold the tech books. No, not Prohibition Orcs–all four of the people who buy those love them, but the sales tell me I’ve done something wrong with those tales. My cozy mystery git commit murder outsold Relayd and Httpd Mastery. But what outdid them both, as well as most of my older books? What title utterly dominated my sales for the last quarter of the year? It was of course, my open source software political satire disguised as porn Savaged by Systemd: an Erotic Unix Encounter. > I can’t believe I just wrote that paragraph. The good news is, once I recovered from EuroBSDCon, my writing got better. I finished Absolute FreeBSD, 3rd edition and submitted it to the publisher. I wrote the second edition of SSH Mastery (no link, because you can’t order it yet.) I’m plowing through git sync murder, the sequel to git commit murder. I don’t get to see the new Star Wars movie until I finish GSM, so hopefully that’ll be this month. All in all, I wrote 480,200 words in 2017. Most of that was after September. It’s annoyingly close to breaking half a million, but after 2016’s scandalous 195,700, I’ll take it. *** ###PG Phriday: Postgres on ZFS > ZFS is a filesystem originally created by Sun Microsystems, and has been available for BSD over a decade. While Postgres will run just fine on BSD, most Postgres installations are historically Linux-based systems. ZFS on Linux has had much more of a rocky road to integration due to perceived license incompatibilities. > As a consequence, administrators were reluctant or outright refused to run ZFS on their Linux clusters. It wasn’t until OpenZFS was introduced in 2013 that this slowly began to change. These days, ZFS and Linux are starting to become more integrated, and Canonical of Ubuntu fame even announced direct support for ZFS in their 16.04 LTS release. > So how can a relatively obscure filesystem designed by a now-defunct hardware and software company help Postgres? Let’s find out! Eddie waited til he finished high school > Old server hardware is dirt cheap these days, and make for a perfect lab for testing suspicious configurations. This is the server we’ll be using for these tests for those following along at home, or want some point of reference:Dell R710 x2 Intel X5660 CPUs, for up to 24 threads 64GB RAM x4 1TB 7200RPM SATA HDDs H200 RAID card configured for Host Bus Adapter (HBA) mode 250GB Samsung 850 EVO SSD> The H200 is particularly important, as ZFS acts as its own RAID system. It also has its own checksumming and other algorithms that don’t like RAID cards getting in the way. As such, we put the card itself in a mode that facilitates this use case. > Due to that, we lose out on any battery-backed write cache the RAID card might offer. To make up for it, it’s fairly common to use an SSD or other persistent fast storage to act both as a write cache, and a read cache. This also transforms our HDDs into hybrid storage automatically, which is a huge performance boost on a budget. She had a guitar and she taught him some chords > First things first: we need a filesystem. This hardware has four 1TB HDDs, and a 250GB SSD. To keep this article from being too long, we’ve already placed GPT partition tables on all the HDDs, and split the SSD into 50GB for the OS, 32GB for the write cache, and 150GB for the read cache. A more robust setup would probably use separate SSDs or a mirrored pair for these, but labs are fair game. They moved into a place they both could afford > Let’s start by getting a performance baseline for the hardware. We might expect peak performance at 12 or 24 threads because the server has 12 real CPUs and 24 threads, but query throughput actually topped out at concurrent 32 processes. We can scratch our heads over this later, for now, we can consider it the maximum capabilities of this hardware. Here’s a small sample: ``` $> pgbench -S -j 32 -c 32 -M prepared -T 20 pgbench...
tps = 264661.135288 (including connections establishing)
tps = 264849.345595 (excluding connections establishing)> So far, this is pretty standard behavior. 260k prepared queries per second is great read performance, but this is supposed to be a filesystem demonstration. Let’s get ZFS involved. + The papers said Ed always played from the heart > Let’s repeat that same test with writes enabled. Once that happens, filesystem syncs, dirty pages, WAL overhead, and other things should drastically reduce overall throughput. That’s an expected result, but how much are we looking at, here?$> pgbench -j 32 -c 32 -M prepared -T 10 pgbench
...
tps = 6153.877658 (including connections establishing)
tps = 6162.392166 (excluding connections establishing)> SSD cache or not, storage overhead is a painful reality. Still, 6000 TPS with writes enabled is a great result for this hardware. Or is it? Can we actually do better? > Consider the Postgres full_page_writes parameter. Tomas Vondra has written about it in the past as a necessity to prevent WAL corruption due to partial writes. The WAL is both streaming replication and crash recovery, so its integrity is of utmost importance. As a result, this is one parameter almost everyone should leave alone. > ZFS is Copy on Write (CoW). As a result, it’s not possible to have a torn page because a page can’t be partially written without reverting to the previous copy. This means we can actually turn off full_page_writes in the Postgres config. The results are some fairly startling performance gains:$> pgbench -j 32 -c 32 -M prepared -T 10 pgbench
tps = 10325.200812 (including connections establishing)
tps = 10336.807218 (excluding connections establishing)> That’s nearly a 70% improvement. Due to write amplification caused by full page writes, Postgres produced 1.2GB of WAL files during a 1-minute pgbench test, but only 160MB with full page writes disabled. > To be fair, a 32-thread pgbench write test is extremely abusive and certainly not a typical usage scenario. However, ZFS just ensured our storage a much lower write load by altering one single parameter. That means the capabilities of the hardware have also been extended to higher write workloads as IO bandwidth is not being consumed by WAL traffic. + They both met movie stars, partied and mingled > Astute readers may have noticed we didn’t change the default ZFS block size from 128k to align with the Postgres default of 8kb. > As it turns out, the 128kb blocks allow ZFS to better combine some of those 8kb Postgres pages to save space. That will allow our measly 2TB to go a lot further than is otherwise possible. > Please note that this is not de-duplication, but simple lz4 compression, which is nearly real-time in terms of CPU overhead. De-duplication on ZFS is currently an uncertain bizzaro universe populated with misshapen horrors crawling along a broken landscape. It’s a world of extreme memory overhead for de-duplication tables, and potential lost data due to inherent conflicts with the CoW underpinnings. Please don’t use it, let anyone else use it, or even think about using it, ever. + They made a record and it went in the chart > We’re still not done. One important aspect of ZFS as a CoW filesystem, is that it has integrated snapshots. > Consider the scenario where a dev is connected to the wrong system and drops what they think is a table in a QA environment. It turns out they were in the wrong terminal and just erased a critical production table, and now everyone is frantic. + The future was wide open > It’s difficult to discount an immediately observable reduction in write overhead. Snapshots have a multitude of accepted and potential use cases, as well. In addition to online low-overhead compression, and the hybrid cache layer, ZFS boasts a plethora of features we didn’t explore. > Built-in checksums with integrated self-healing suggest it isn’t entirely necessary to re-initialize an existing Postgres instance to enable checksums. The filesystem itself ensures checksums are validated and correct, especially if we have more than one drive resource in our pool. It even goes the extra mile and actively corrects inconsistencies when encountered. > I immediately discounted ZFS back in 2012 because the company I worked for at the time was a pure Linux shop. ZFS was only available using the FUSE driver back then, meaning ZFS only worked through userspace with no real kernel integration. It was fun to tinker with, but nobody sane would use that on a production server of any description. > Things have changed quite drastically since then. I’ve stopped waiting for btrfs to become viable, and ZFS has probably taken the throne away from XFS as my filesystem of choice. Future editions of the Postgres High Availability Cookbook will reflect this as well. > Postgres MVCC and ZFS CoW seem made for each other. I’m curious to see what will transpire over the next few years now that ZFS has reached mainstream acceptance in at least one major Linux distribution. *** ###[NomadBSD](https://github.com/mrclksr/NomadBSD) + About > NomadBSD is a live system for flash drives, based on FreeBSD. + Screenshots   + Requirements for building the image A recent FreeBSD system + Requirements for running NomadBSD * A 4GB (or more) flash drive * A System capable running FreeBSD 11.1 (amd64) + Building the image ~~ csh # make image ~~ + Writing the image to an USB memory stick ~~ csh # dd if=nomadbsd.img of=/dev/da0 bs=10240 conv=sync ~~ + Resize filesystem to use the entire USB memory + Boot NomadBSD into single user mode, and execute: ~~ # gpart delete -i 2 da0s1 # gpart resize -i 1 da0 # gpart commit da0s1 ~~ + Determine the partition size in megabytes using ````fdisk da0```` + and calculate the remaining size of da0s1a: ````<REMAIN> = <SIZE OF PARTITION IN MB> - <DESIRED SWAP SIZE IN MB>````. ~~ # gpart resize -i 1 -s <REMAIN>M da0s1 # gpart add -t freebsd-swap -i 2 da0s1 # glabel label NomadBSDsw da0s1b # service growfs onestart # reboot ~~ + [FreeBSD forum thread](https://forums.freebsd.org/threads/63888/) + [A short screen capture video of the NomadBSD system running in VirtualBox](https://freeshell.de/~mk/download/nomad_capture.mp4) *** ##Beastie Bits + [Coolpkg, a package manager inspired by Nix for OpenBSD](https://github.com/andrewchambers/coolpkg) + [zrepl - ZFS replication](https://zrepl.github.io/) + [OpenBSD hotplugd automount script](https://bijanebrahimi.github.io/blog/openbsd-hotplugd-scripting.html) + [Ancient troff sources vs. modern-day groff](https://virtuallyfun.com/2017/12/22/learn-ancient-troff-sources-vs-modern-day-groff/) + [Paypal donation balance and status.. thanks everyone!](http://lists.dragonflybsd.org/pipermail/users/2017-December/313752.html) + [Supervised FreeBSD rc.d script for a Go daemon (updated in last few days)](https://redbyte.eu/en/blog/supervised-freebsd-init-script-for-go-deamon/) + [A Brief History of sed](https://blog.sourcerer.io/a-brief-history-of-sed-6eaf00302ed) + [Flamegraph: Why does my AWS instance boot so slow?](http://www.daemonology.net/timestamping/tslog-c5.4xlarge.svg) *** ##Feedback/Questions + Jeremy - [Replacing Drive in a Zpool](http://dpaste.com/319593M#wrap) + [Dan’s Blog ](https://dan.langille.org/2017/08/16/swapping-5tb-in-3tb-out/) + Tim - [Keeping GELI key through reboot](http://dpaste.com/11QTA06) + Brian - [Mixing 2.5 and 3.5 drives](http://dpaste.com/2JQVD10#wrap) + Troels - [zfs swap on FreeBSD](http://dpaste.com/147WAFR#wrap) ***Wed, 10 Jan 2018 - 1h 51min - 827 - 227: The long core dump
We walk through dumping a PS4 kernel in only 6 days, tell you the news that NetBSD 7.1.1 has been released, details on how to run FreeBSD on a Thinkpad T470, and there’s progress in OpenBSD’s pledge.
This episode was brought to you by
Headlines
NetBSD 7.1.1 released
The NetBSD Project is pleased to announce NetBSD 7.1.1, the first security/critical update of the NetBSD 7.1 release branch. It represents a selected subset of fixes deemed important for security or stability reasons. Complete source and binaries for NetBSD 7.1.1 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, and other services may be found at https://www.NetBSD.org/mirrors/. We encourage users who wish to install via ISO or USB disk images to download via BitTorrent by using the torrent files supplied in the images area. A list of hashes for the NetBSD 7.1.1 distribution has been signed with the well-connected PGP key for the NetBSD Security Officer: https://ftp.NetBSD.org/pub/NetBSD/security/hashes/NetBSD-7.1.1_hashes.asc NetBSD is free. All of the code is under non-restrictive licenses, and may be used without paying royalties to anyone. Free support services are available via our mailing lists and website. Commercial support is available from a variety of sources. More extensive information on NetBSD is available from our website: [NetBSD website](www.NetBSD.org) +Changes Between 7.1 and 7.1.1 Below is an abbreviated list of changes in this release. The complete list can be found in the CHANGES-7.1.1 file in the top level directory of the NetBSD 7.1.1 release tree. Security Advisory Fixes The following security advisories were fixed: NetBSD-SA2017-004 buffer overflow via cmap for 4 graphics drivers. NetBSD-SA2017-005 x86: vulnerabilities in context handling. NetBSD-SA2017-006 Vnode reference leak in the openat system call. NetBSD-SA2018-001 Several vulnerabilities in context handling NetBSD-SA2018-002 Local DoS in virecover Note: Advisories prior to NetBSD-SA2017-004 do not affect NetBSD 7.1.1. Userland changes dhcrelay(8): Fix bug that prevented proper operation when run in the background. Heimdal: Update to 7.1. Fix CVE-2017-11103. mtree(8): Don't modify strings stored in hash, otherwise filling up of directory hierarchy stops if the same hash value occurs in directory and leaf. ping(8): Fix cksum calculation for clearing the cached route. resize_ffs(8): Fix numerous overflow errors which can lead to superblock corruption on large filesystems. rtadvd(8): Fix the default value of rltime. PR bin/51994. Update BIND to 9.10.5-P2. Update expat to 2.2.1. Update ntp to 4.2.8p10. Update root.cache to 2017102400. Update tzdata to 2017c. vi(1): Don't garble display when when resizing nvi in xterm. wpa_supplicant/hostapd: Update to 2.6. Apply fixes for CVEs 2017-13077 through 2017-13082 and CVEs 2017-13086 through 2017-13088. X: Apply fixes for CVEs 2017-12176 through 2017-12187, 2017-10971, 2017-10972, 2017-13722, 2017-13720, 2017-16611, and 2017-16612. *** ###Dumping a PS4 Kernel in "Only" 6 Days > What if a secure device had an attacker-viewable crashdump format? What if that same device allowed putting arbitrary memory into the crashdump? Amazingly, the ps4 tempted fate by supporting both of these features! Let’s see how that turned out… Crashdumps on PS4 The crash handling infrastructure of the ps4 kernel is interesting for 2 main reasons: It is ps4-specific code (likely to be buggy) If the crashdump can be decoded, we will gain very useful info for finding bugs and creating reliable exploits On a normal FreeBSD system, a kernel panic will create a dump by calling kern_reboot with the RB_DUMP flag. This then leads to doadump being called, which will dump a rather tiny amount of information about the kernel image itself to some storage device. On ps4, the replacement for doadump is mdbg_run_dump, which can be called from panic or directly from trap_fatal. The amount of information stored into the dump is gigantic by comparison - kernel state for all process, thread, and vm objects are included, along with some metadata about loaded libraries. Other obvious changes from the vanilla FreeBSD method are that the mdbg_run_dump encodes data recorded into the dump on a field-by-field basis and additionally encrypts the resulting buffer before finally storing it to disk. Dumping Anything Let’s zoom in to a special part of mdbg_run_dump - where it iterates over all process’ threads and tries to dump some pthread state: dumpstate is a temporary buffer which will eventually make it into the crashdump. To summarize, sysdump__internal_call_readuser can be made to function as a read-anywhere oracle. This is because fsbase will point into our (owned) webkit process’ usermode address space. Thus, even without changing the actual fsbase value, we may freely change the value of tcb_thread, which is stored at fsbase + 0x10. Further, sysdump__internal_call_readuser will happily read from a kernel address and put the result into the dump. We can now put any kernel location into the dump, but we still need to decrypt and decode it… Aside from that, there’s also the issue that we may only add 0x10 bytes per thread in this manner… Further reading: Crashdump Crypto Crashdump Decoding Crashdump Automation Triggering the Vulnerability The Fix (Kind of…) Fin Appendix Crashdump Decryptor NXDP Decoder *** ###BSDTW 2017 Conference Recap: Li-Wen Hsu BSDTW 2017 Conference Recap: Li-Wen Hsu 12/28/2017 > Last month, we held BSDTW 2017 on November 11-12th, 2017 in Taipei, Taiwan. It was the second largest BSD conference in Taiwan and the first one in this decade. In 2004, the first AsiaBSDCon was also held in Taipei. Then all of the following AsiaBSDCon conferences were held in Tokyo, Japan. (AsiaBSDCon 2018 will be in Tokyo again next year, please submit your talk proposal by December 31th 2017, and attend the conference on March 8th-11th) > We wanted to start small with the first BSDTW because we were not sure how much sponsorship or how many volunteers we might have. BSDTW 2017 was a single track, two-day conference with 11 selected 50 minute presentations and 1 WIP/lightning talk session consisting of 8 short talks. I do regret that we did not have any local presenters this year. It is also a similar problem at AsiaBSDCon. Unsurprisingly, as with AsiaBSDcon, the travel reimbursement took up a large part of the whole conference budget. We do have many good people that work in Asia, but we still need to encourage people to present their work more. > We had over 130 registered attendees, with 30% of them coming from outside of Taiwan. To our knowledge, in recent years, this is the only open source conference in Taiwan to be held entirely in English, and to have such a large portion of international attendees. This is also the first open source conference in Taiwan to focus entirely on operating systems. The attendees included students, professors, engineers or CTOs, and CEOs from technology companies. This is also the first time that GroffTheBSDGoat visited Taiwan! We were surprised that after the silence for so many years, there are still so many people that use and love BSD near us. We saw many old friends, who had “disappeared” for a long time, came back, and were glad to meet many new friends at the conference. I am really happy that this conference was able to bring together these people, from local and abroad. After attending BSD conferences around the world for many years, I feel that the friendship between BSD users is the most important thing in the BSD community, and one of the main reasons people stay. It has been my pleasure to bring this community back to my friends in my homeland. > After the two-day event, I truly understand that bootstrapping a new conference is a very hard job. One with many aspects that you don’t even imagine until you’re really in the process of planning an event. I now have an even greater respect for all of the conference organizers and realize that we need to have more people help them, to keep these conferences continue to get better and better. Plus, there will always be room for a new conference! > Thanks to the FreeBSD Foundation for being the biggest sponsor of BSDTW 2017 and always being the strongest backend of our community. We are excited about the many local companies and organizations that helped us whether with people, materials or financially. We even had 21 personal sponsors, more than two times the number of other big open source conferences in Taiwan. > As I said in the closing session, I’m not sure if there will be 2nd BSDTW next year. It still depends on the amount of sponsorship and number of volunteers. However, we will definitely hold more smaller meetups in the next year to keep building up the local BSD community. > Finally, in the beginning of this month, we had a “post-conference media workshop” for organizing the media files we collected in the BSDTW 2017. Here are the review article in Traditional Chinese and the photos: https://medium.com/@bsdtw/bsdtw-2017-總回顧-a402788daede && https://www.flickr.com/photos/bsdtw/albums/72157689410035911 *** ##News Roundup ###Running FreeBSD on a Lenovo T470s Running FreeBSD on the Lenovo T470s ThinkPad > Installing FreeBSD on this machine was super easy. As I couldn't find a comprehensive/encouraging how-to about installing FreeBSD on a recent ThinkPad, I just wrote up the one below. It includes details about my personal setup, which are not required to run FreeBSD on this model, but which are more to my own taste. I still think this can be a quite useful inspiration for others who want to run their own customized configurations. Specs > The system I use has these specifications: Type: 20JS-001EGE CPU: Intel Core i7-6600U, 2x 2.60GHz RAM: 20GB DDR4 SSD: 512GB NVMe Graphics: Intel HD Graphics 520 (IGP), 1x HDMI 1.4 Display: 14", 1920x1080, non-glare, IPS Ports: 3x USB-A 3.0, 1x Thunderbolt 3, 1x Gb LAN Wireless: WLAN 802.11a/b/g/n/ac, Bluetooth 4.1, LTE (Micro-SIM) Cardreader: SD/SDHC/SDXC/MMC Webcam: 0.9 Megapixel Extras: MIL-STD-810G, Pointing Stick, Fingerprint-Reader, Docking port Things that work > Basically everything I care about: Accelerated video Keyboard Touchpad/ClickPad (like expected in a modern laptop) SSD WiFi Sound HDMI out Suspend to RAM Webcam Things that don't work Fingerprint reader Potentially anything I didn't test Battery life is okay, but could be better. Installation of the base system > I used a snapshot release of 12-CURRENT as the basis of my installation, particularly the one of 13th of December 2017. > I dd'ed it onto a memory stick and boot the laptop. I started a standard installation and created an encrypted ZFS pool on nvme0, using encryption, swap encryption and partition scheme "GPT (UEFI)". > After installation, it boots straight up. Ports tree used > All work is based on a head ports tree from about Dec 18, 22:15 CET, which should be more or less r456672. Preferred ClickPad configuration > As I'm not a fan of the the pointing stick, I disabled it in the bios. My final ClickPad configuration will be: Click to click (not tap), no middle button, right button in the lower right corner. As the old synaptics driver doesn't provide good thumb detection, libinput will be used. Check out the laptop list on the FreeBSD wiki for compatibility: (https://wiki.freebsd.org/Laptops/) *** ###FreeBSD desktop LiveCD creator Introduction > The purpose of this tool is quickly generate bloat free images containing stock FreeBSD, and supported desktop environments. Features FreeBSD 11.1-RELEASE AMD64 Gnome & KDE desktop environments Hybrid DVD/USB image Screenshots [Gnome LiveCD])https://github.com/pkgdemon/comet/raw/master/screenshots/gnome-livecd.png?raw=true) KDE LiveCD System Requirements FreeBSD 11.1, or higher for AMD64 20GB of free disk space 1GB of free memory UFS, or ZFS Initial Setup Install the required packages:pkg install git grub2-pcbsd grub2-efi xorrisoClone the repo:git clone https://www.github.com/pkgdemon/cometEnter the directory for running the LiveCD creator:cd comet/srcCredentials for live media > User: liveuser > Password: freebsd *** ###iXsystems StorageCrypter Ransomware: Security Threat or Clickbait? ###pledge() work in progress > I wanted to give an update that a two pledge-related changes are being worked on. The semantics and integration are complicated so it is taking some time. > One is execpromises. This will become the 2nd argument of pledge(). This allows one to set the pledge for the new image after pledge "exec"-allowed execve(). A warning though: utilizing this in software isn't as easy as you might think! The fork+exec + startup sequences needed to be studied quite carefully to ensure the newly-executed child doesn't ask for more than the parent's execpromises. In my experiments such a circumstance is exceedingly common, so the problem is eased by introducing a new pledge feature which allows pledge violations to return ENOSYS or such rather than killing the process. > This feature also needs to be used with great caution (especially in privileged programs) because programs which fail to observe errors may continue operating forward very incorrectly; you've lost the ability to catch it failing, and provide care by fixing the problem. > The other is pledgepaths. The semantics are still being tuned a bit. Before the first call to pledge() in a process, one can pledgepath() directories. Then later after pledge(), file access operations only work if the traversal of the path crosses one of those pre-declared directories (but better make sure you don't move a directory, because the kernel remembers and reasons about the vnode of the directory rather than the path). Something similar is being worked on for files, but we are still adjusting that, as well as a flag parameter for the pledgepath() call which may constrain the operations done on such files. > As such, pledgepath() will become a filesystem containment mechanism unlike chroot() because paths will still be based upon true /. > Patience. *** ###The anatomy of tee program on OpenBSD > The tee command is used to read content from standard input and displays it not only in standard output but also saves to other files simultaneously. The source code of tee in OpenBSD is very simple, and I want to give it an analysis: > (1) tee leverages Singlely-linked List defined in sys/queue.h to manage outputted files (including standard output):struct list { SLIST_ENTRY(list) next; int fd; char *name; }; SLIST_HEAD(, list) head; ...... static void add(int fd, char *name) { struct list *p; ...... SLIST_INSERT_HEAD(&head, p, next); } int main(int argc, char *argv[]) { struct list *p; ...... SLIST_INIT(&head); ...... SLIST_FOREACH(p, &head, next) { ...... } }> To understand it easily, I extract the macros from sys/queue.h and created a file which utilizes the marcos:#define SLIST_HEAD(name, type) \ struct name { \ struct type *slh_first; /* first element */ \ } #define SLIST_ENTRY(type) \ struct { \ struct type *sle_next; /* next element */ \ } #define SLIST_FIRST(head) ((head)->slh_first) #define SLIST_END(head) NULL #define SLIST_EMPTY(head) (SLIST_FIRST(head) == SLIST_END(head)) #define SLIST_NEXT(elm, field) ((elm)->field.sle_next) #define SLIST_FOREACH(var, head, field) \ for((var) = SLIST_FIRST(head); \ (var) != SLIST_END(head); \ (var) = SLIST_NEXT(var, field)) #define SLIST_INIT(head) { \ SLIST_FIRST(head) = SLIST_END(head); \ } #define SLIST_INSERT_HEAD(head, elm, field) do { \ (elm)->field.sle_next = (head)->slh_first; \ (head)->slh_first = (elm); \ } while (0) struct list { SLIST_ENTRY(list) next; int fd; char *name; }; SLIST_HEAD(, list) head; int main(int argc, char *argv[]) { struct list *p; SLIST_INIT(&head); SLIST_INSERT_HEAD(&head, p, next); SLIST_FOREACH(p, &head, next) { } }> Then employed gcc‘s pre-processing function:# gcc -E slist.c # 1 "slist.c" # 1 "<built-in>" # 1 "<command-line>" # 1 "slist.c" # 30 "slist.c" struct list { struct { struct list *sle_next; } next; int fd; char *name; }; struct { struct list *slh_first; } head; int main(int argc, char *argv[]) { struct list *p; { ((&head)->slh_first) = NULL; }; do { (p)->next.sle_next = (&head)->slh_first; (&head)->slh_first = (p); } while (0); for((p) = ((&head)->slh_first); (p) != NULL; (p) = ((p)->next.sle_next)) { } }> It becomes clear now! The head node in list contains only 1 member: slh_first, which points to the first valid node. For the elements in the list, it is embedded with next struct which uses sle_next to refer to next buddy. > (2) By default, tee will overwrite the output files. If you want to append it, use -a option, and the code is as following:while (*argv) { if ((fd = open(*argv, O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC), DEFFILEMODE)) == -1) { ...... } ...... }> (3) The next part is the skeleton of saving content to files:while ((rval = read(STDIN_FILENO, buf, sizeof(buf))) > 0) { SLIST_FOREACH(p, &head, next) { n = rval; bp = buf; do { if ((wval = write(p->fd, bp, n)) == -1) { ...... } bp += wval; } while (n -= wval); } }> We need to iterates every opened file descriptor and write contents into it. > (4) Normally, theinterrupt signal will cause tee exit:# tee fdkfkdfjk fdkfkdfjk ^C #> To disable this feature, use -i option:# tee -i fdhfhd fdhfhd ^C^C> The corresponding code is like this:...... case 'i': (void)signal(SIGINT, SIG_IGN); break;*** ##Beastie Bits + What I learned from reading the OpenBSD's network stack source code + Broadcom BCM43224 and BCM43225 Wi-Fi cards now supported by bwn(4) + Ingo details searching man pages + DTrace & ZFS Being Updated On NetBSD, Moving Away From Old OpenSolaris Code + Linux Professional Institute and BSD Certification Group Join Efforts + The FreeBSD Foundation thanks Donors ##Feedback/Questions + Alex - My first freebsd bug + John - Suggested Speakers + Todd - Two questions + Matthew - CentOS to FreeBSD ***Wed, 03 Jan 2018 - 1h 31min - 826 - 226: SSL: Santa’s Syscall List
We read the FreeBSD Q3 status report, explore good and bad syscalls, list GOG Games for OpenBSD, and show you what devmatch can do.
This episode was brought to you by
Headlines
FreeBSD Q3 Status Report 2017
FreeBSD Team Reports FreeBSD Release Engineering Team Ports Collection The FreeBSD Core Team The FreeBSD Foundation Projects FreeBSD CI Kernel Intel 10G iflib Driver Update Intel iWARP Support pNFS Server Plan B Architectures AMD Zen (family 17h) support Userland Programs Updates to GDB Ports FreeBSDDesktop OpenJFX 8 Puppet Documentation Absolute FreeBSD, 3rd Edition Manual Pages Third-Party Projects The nosh Project ####FreeBSD Foundation Q4 Update *** ###11 syscalls that rock the world 0. read > You cannot go wrong with a read. You can barely EFAULT it! On Linux amd64 it is syscall zero. If all its arguments are zero it returns zero. Cool! 1. pipe > The society for the preservation of historic calling conventions is very fond of pipe, as in many operating systems and architectures it preserves the fun feature of returning both of the file descriptors as return values. At least Linux MIPS does, and NetBSD does even on x86 and amd64. Multiple return values are making a comeback in languages like Lua and Go, but C has always had a bit of a funny thing about them, but they have long been supported in many calling conventions, so let us use them in syscalls! Well, one syscall. 2. kqueue > When the world went all C10K on our ass, and scaleable polling was a thing, Linux went epoll, the BSDs went kqueue and Solaris went /dev/poll. The nicest interface was kqueue, while epoll is some mix of edge and level triggered semantics and design errors so bugs are still being found. 3. unshare > Sounds like a selfish syscall, but this generous syscall call is the basis of Linux namespaces, allowing a process to isolate its resources. Containers are built from unshares. 4. setns > If you liked unshare, its younger but cooler friend takes file descriptors for namespaces. Pass it down a unix socket to another process, or stash it for later, and do that namespace switching. All the best system calls take file descriptors. 5. execveat > Despite its somewhat confusing name (FreeBSD has the saner fexecve, but other BSDs do not have support last time I checked), this syscall finally lets you execute a program just given a file descriptor for the file. I say finally, as Linux only implemented this in 3.19, which means it is hard to rely on it (yeah, stop using those stupid old kernels folks). Before that Glibc had a terrible userspace implementation that is basically useless. Perfect for creating sandboxes, as you can sandbox a program into a filesystem with nothing at all in, or with a totally controlled tree, by opening the file to execute before chroot or changing the namespace. 6. pdfork > Too cool for Linux, you have to head out to FreeBSD for this one. Like fork, but you get a file descriptor for the process not a pid. Then you can throw it in the kqueue or send it to another process. Once you have tried process descriptors you will never go back. 7. signalfd > You might detect a theme here, but if you have ever written traditional 1980s style signal handlers you know how much they suck. How about turning your signals into messages that you can read on, you guessed it, file descriptors. Like, usable. 8. wstat > This one is from Plan 9. It does the opposite of stat and writes the same structure. Simples. Avoids having chmod, chown, rename, utime and so on, by the simple expedient of making the syscall symmetric. Why not? 9. clonefile > The only cool syscall on OSX, and only supported on the new APFS filesystem. Copies whole files or directories on a single syscall using copy on write for all the data. Look on my works, copy_file_range and despair. 10. pledge > The little sandbox that worked. OpenBSD only here, they managed to make a simple sandbox that was practical for real programs, like the base OpenBSD system. Capsicum form FreeBSD (and promised for Linux for years but no sign) is a lovely design, and gave us pdfork, but its still kind of difficult and intrusive to implement. Linux has, well, seccomp, LSMs, and still nothing that usable for the average program. ###Eleven syscalls that suck 0. ioctl > It can‘t decide if it‘s arguments are integers, strings, or some struct that is lost in the midst of time. Make up your mind! Plan 9 was invented to get rid of this. 1. fcntl > Just like ioctl but for some different miscellaneous operations, because one miscelleny is not enough. 2. tuxcall > Linux put a web server in the kernel! To win a benchmark contest with Microsoft! It had it‘s own syscall! My enum tux_reactions are YUK! Don‘t worry though, it was a distro patch (thanks Red Hat!) and never made it upstream, so only the man page and reserved number survive to taunt you and remind you that the path of the righteous is beset by premature optmization! 3. io_setup > The Linux asynchronous IO syscalls are almost entirely useless! Almost nothing works! You have to use O_DIRECT for a start. And then they still barely work! They have one use, benchmarking SSDs, to show what speed you could get if only there was a usable API. Want async IO in kernel? Use Windows! 4. stat, and its friends and relatives > Yes this one is useful, but can you find the data structure it uses? We have oldstat, oldfstat, ustat, oldlstat, statfs, fstatfs, stat, lstat, fstat, stat64, lstat64, fstat64, statfs64, fstatfs64, fstatat64 for stating files and links and filesystems in Linux. A new bunch will be along soon for Y2038. Simplify your life, use a BSD, where they cleaned up the mess as they did the cooking! Linux on 32 bit platforms is just sucky in comparison, and will get worse. And don’t even look at MIPS, where the padding is wrong. 5. Linux on MIPS > Not a syscall, a whole implemntation of the Linux ABI. Unlike the lovely clean BSDs, Linux is different on each architecture, system calls randomly take arguments in different orders, and constants have different values, and there are special syscalls. But MIPS takes the biscuit, the whole packet of biscuits. It was made to be binary compatible with old SGI machines that don’t even exist, and has more syscall ABIs than I have had hot dinners. Clean it up! Make a new sane MIPS ABI and deprecate the old ones, nothing like adding another variant. So annoying I think I threw out all my MIPS machines, each different. 6. inotify, fanotify and friends > Linux has no fewer than three file system change notification protocols. The first, dnotify hopped on ioctl‘s sidekick fcntl, while the two later ones, inotify and fanotify added a bunch more syscalls. You can use any of them, and they still will not provide the notification API you want for most applications. Most people use the second one, inotify and curse it. Did you know kqueue can do this on the BSDs? 7. personality > Oozing in personality, but we just don’t get along. Basically obsolete, as the kernel can decide what kind of system emulation to do from binaries directly, it stays around with some use cases in persuading ./configure it is running on a 32 bit system. But it can turn off ASLR, and let the CVEs right into your system. We need less persoanlity! 8. gettimeofday > Still has an obsolete timezone value from an old times when people thought timezones should go all the way to the kernel. Now we know that your computer should not know. Set its clock to UTC. Do the timezones in the UI based on where the user is, not the computer. You should use clock_gettime now. Don’t even talk to me about locales. This syscall is fast though, don’t use it for benchmarking, its in the VDSO. 9. splice and tee > These, back in 2005 were a quite nice idea, although Linux said then “it is incomplete, the interfaces are ugly, and it will oops the system if anything goes wrong”. It won’t oops your system now, but usage has not taken off. The nice idea from Linus was that a pipe is just a ring buffer in the kernel, that can have a more general API and use cases for performant code, but a decade on it hasn’t really worked out. It was also supposed to be a more general sendfile, which in many ways was the successor of that Tux web server, but I think sendfile is still more widely used. 10. userfaultfd > Yes, I like file descriptors. Yes CRIU is kind of cool. But userspace handling page faults? Is nothing sacred? I get that you can do this badly with a SIGSEGV handler, but talk about lipstick on a pig. *** ###OpenBSD 6.0 on an iMac G3 from 1999 > A while ago I spent $50 for an iMac G3 (aka the iMac,1). This iconic model restored Apple's fortunes in the late '90s. Since the iMac G3 can still boot Mac OSes 8 and 9, I mostly use the machine to indulge a nostalgia for childhood schooldays spent poking at the operating system and playing Escape Velocity. But before I got around to that, I decided to try out the software that the previous owner had left on the machine. The antiquated OSX 10.2 install and 12 year old versions of Safari and Internet Explorer were too slow and old to use for anything. Updating to newer software was almost impossible; a later OSX is required to run the little PowerPC-compatible software still languishing in forgotten corners of the Internet. This got me thinking: could this machine be used, really used, nowadays? Lacking a newer OSX disc, I decided to try the most recent OpenBSD release. (And, since then, to re-try with each new OpenBSD release.) Below are the results of this experiment (plus a working xorg.conf file) and a few background notes. Background > This iMac is a Revision D iMac G3 in grape. It's part of the iMac,1 family of computers. This family includes all tray-loading iMac G3s. (Later iMac G3s had a slot-loading CD drive and different components.) Save for a slightly faster processor, a dedicated graphics card, and cosmetic tweaks to the case, my iMac is identical to the prior year's line-launching Bondi Blue iMac. My machine has had its memory upgraded from 32 MB to 320 MB. Thank Goodness. > The Revision D iMac G3 shipped with Mac OS 8.5. It can run up to Mac OS 9.2.2 or OSX 10.3.9. Other operating systems that tout support for the iMac,1 include NetBSD, OpenBSD, and a shrinking number of Linux distributions. > OpenBSD is simple (by design) and well-maintained. In contrast, NetBSD seems rather more complex and featureful, and I have heard grumbling that despite its reputation for portability, NetBSD really only works well on amd64. I'd test that assertion if OpenBSD's macppc installation instructions didn't seem much simpler than NetBSD's. Linux is even more complicated, although most distros are put together in a way that you can mostly ignore that complexity (until you can't). In the end I went with OpenBSD because I am familiar with it and because I like it. Installing OpenBSD on the iMac,1 > Installing OpenBSD on this iMac was simple. It's the same procedure as installing OpenBSD on an amd64 rig. You put in the installation disc; you tell the machine to boot from it; and then you answer a few prompts, most of which simply ask you to press enter. In this case, OpenBSD recognizes all machine's hardware just fine, including sound and networking, though I had a little trouble with video. > The OpenBSD documentation says video should just work and that an xorg.conf file isn't necessary. As such, it no longer ships with an xorg.conf file. Though that's never posed a problem on my other OpenBSD machines, it does here. Video doesn't work out of the box on my iMac,1. startx just blanks the screen. Fortunately, because the BSDs use a centralized development model where each operating system is stored in one repository, OpenBSD's website provides a web interface to the source code going back to the early days. I was able to find the last version of the sample xorg.conf that used to ship on macppc. With a little tweaking, I transformed that file into this one, with which video works just fine. Just drop it into your iMac's /etc/X11 directory. You'll also need to remember to set the machdep.allowaperture sysctl to 2 (e.g., as root run sysctl machdep.allowaperture=2), although the installer will do that automatically if you answer yes to the question about whether you plan to run X. > All that being said, video performance is pretty poor. I am either doing something wrong, or OpenBSD doesn't have accelerated video for this iMac, or this machine is just really old! I will discuss performance below. Running OpenBSD on the iMac,1 > The machine performs okay under OpenBSD. You can expect to ably run minimalistic software under minimalistic window managers. I tried dillo, mrxvt, and cmus under cwm and fvwm. Performance here was just fine. I also tried Firefox 26, 33, and 34 under fvwm and cwm. Firefox ran, but "modern," Javascript-heavy sites were an exercise in frustration; the 2015 version of CNN.com basically froze Firefox for 30 seconds or more. A lighter browser like dillo is doable. > You'll notice that I used the past-tense to talk about Firefox. Firefox currently doesn't build on PowerPC on OpenBSD. Neither does Chromium. Neither do a fair number of applications. But whatever -- there's still a lot of lighter applications available, and it's these you'll use day-to-day on a decades-old machine. > Lightweight window managers work okay, as you'd expect. You can even run heavier desktop environments, such as xfce, though you'll give up a lot of performance. > I ran the Ubench benchmark on this iMac and two more modern machines also running OpenBSD. The benchmark seems like an old one; I don't know how (if at all) it accounts for hardware changes in the past 13 years. That is, I don't know if the difference in score accurately measures the difference in real-world performance. Here are the results anyway: Conclusion > Except for when I check to see if OpenBSD still works, I run Mac OS9 on this rig. I have faster and better machines for running OpenBSD. If I didn't -- if this rig were, improbably, all I had left, and I was waiting on the rush delivery of something modern -- then I would use OpenBSD on my iMac,1. I'd have to stick to lightweight applications, but at least they'd be up-to-date and running on a simple, stable, OS. *** ##News Roundup ###34th Chaos Communication Congress Schedule Many talks are streamed live, a good mixture of english and german talks May contain DTraces of FreeBSD Are all BSDs created equally? library operating systems Hardening Open Source Development *** ###OpenBSD 6.2 + CDE > If you've noticed a disruption in the time-space continuum recently, it is likely because I have finally been able to compile and install the Common Desktop Environment (CDE) in a current and actively-developed operating system (OpenBSD 6.2 in this case). > This comes after so many attempts (across multiple platforms) that ended up with the build process prematurely stopping itself in its own tracks for a variety of infinitesimal reasons that were beyond my comprehension as a non-programmer, or when there was success it was not without some broken parts. As for the latter, I've been able to build CDE on OpenIndiana Hipster, but with an end product where I'm unable to change the color scheme in dtstyle (because "useColorObj" is set to "False"), with a default color scheme that is low-res and unpleasant. As for changing "useColorObj" to "True", I tried every recommended trick I could find online, but nothing worked. > My recent attempts at installing CDE on OpenBSD (version 6.1) saw the process stop due to a number of errors that are pure gibberish to these naive eyes. While disappointing, it was par for the course within my miserable experience with trying to build this particular desktop environment. As I wrote in this space in November 2015, in the course of explaining part of my imperitive for installing Solaris 10: > And so I have come to think of building the recently open-sourced CDE as being akin to a coffee mug I saw many years ago. One side of the mug read "Turn the mug to see how to keep an idiot busy." On the other side, it read "Turn the mug to see how to keep an idiot busy." I'm through feeling like an idiot, which is partially why I'm on this one-week journey with Solaris 10. > While I thoroughly enjoyed running Solaris 10 on my ThinkPad T61p, and felt a devilish thrill at using it out in the open at my local MacBook- and iPhone-infested Starbucks and causing general befuddlement and consternation among the occasional prying yoga mom, I never felt like I could do much with it beyond explore the SunOS 5.10 command line and watch YouTube videos. While still supported by its current corporate owner (whose name I don't even want to type), it is no longer actively developed and is thus little more than a retro toy. I hated the idea of installing anything else over it, but productivity beckoned and it was time to tearfully and reluctantly drag myself off the dance floor. > In any case, just last week I noticed that the Sourceforge page for the OpenBSD build had some 6.2-specific notes by way of a series of four patches, and so I decided 'what the heck, let's give this puppy another whirl'. After an initial abortive attempt at a build, I surmised that I hadn't applied the four patches correctly. A day or two later, I took a deep breath and tried again, this time resolving to not proceed with the time make World build command until I could see some sign of a successful patch process. (This time around, I downloaded the patches and moved them into the directory containing the CDE makefiles, and issued each patch command as patch Once I had the thing up and running, and with a mind bursting with fruit flavor, I started messing about. The first order of business was to create a custom color scheme modelled after the default color scheme in UnixWare. (Despite any baggage that system carries from its previous ownership under SCO, I adored the aesthetics of UnixWare 7.1.4 two years ago when I installed the free one month trial version on my ThinkPad. For reasons that escape me now, I named my newly-created color scheme in honor of UnixWare 7.1.3.) > Like a proud papa, I immediately tweeted the above screenshot and risked irritating a Linux kid or two in the process, given SCO's anti-climatic anti-Linux patent trolling from way back when. (I'm not out to irritate penguinistas, I just sure like this color scheme.) Final Thoughts > It may look a little clunky at first, and may be a little bling-challenged, but the more I use CDE and adapt to it, the more it feels like an extension of my brain. Perhaps this is because it has a lot zip and behaves in a consistent and coherent manner. (I don't want to go too much further down that road here, as OSnews's Thom Holwerda already gave a good rundown about ten years ago.) > Now that I have succesfully paired my absolute favorite operating system with a desktop environment that has exerted an intense gravitational hold on me for many, many years, I don't anticipate distrohopping any time soon. And as I attain a more advanced knowledge of CDE, I'll be chronicling any new discoveries here for the sake of anyone following me from behind as I feel my way around this darkened room. *** ###devmatch(8) added to FreeBSD HEAD ``` Log: Match unattached devices on the system to potential kernel modules.devmatch(8) matchs up devices in the system device tree with drivers
that may match them. For each unattached device in the system, it
tries to find matching PNP info in the linker hints and prints modules
to load to claim the devices.In --unbound mode, devmatch can look for drivers that have attached to
devices in the device tree and have plug and play information, but for
which no PNP info exists. This helps find drivers that haven't been
converted yet that are in use on this system.In addition, the ability to dump out linker.hints is provided.
Future commits will a
Wed, 27 Dec 2017 - 2h 07min - 825 - 225: The one true OS
TrueOS stable 17.12 is out, we have an OpenBSD workstation guide for you, learnings from the PDP-11, FreeBSD 2017 Releng recap and Duo SSH.
This episode was brought to you by
Headlines
TrueOS stable release 17.12
Sane service management and status reporting with OpenRC Reliable, non-interactive system update mechanism with fail-safe boot environment support. Graphical management of remote TrueOS servers through SysAdm (also provides a reliable API for administrating systems remotely). LibreSSL for all base SSL support. Base system managed via packages (allows for additional fine-tuning). Base system is smaller due to the removal of the old GCC version in base. Any compiler and/or version may be installed and used via packages as desired. Support for newer graphics drivers and chipsets (graphics, networking, wifi, and more)We are pleased to announce a new release of the 6-month STABLE version of TrueOS!
This release cycle focused on lots of cleanup and stabilization of the distinguishing features of TrueOS: OpenRC, boot speed, removable-device management, SysAdm API integrations, Lumina improvements, and more. We have also been working quite a bit on the server offering of TrueOS, and are pleased to provide new text-based server images with support for Virtualization systems such as bhyve! This allows for simple server deployments which also take advantage of the TrueOS improvements to FreeBSD such as:
Notable Changes: Over 1100 OpenRC services have been created for 3rd-party packages. This should ensure the functionality of nearly all available 3rd-party packages that install/use their own services. The OpenRC services for FreeBSD itself have been overhauled, resulting in significantly shorter boot times. Separate install images for desktops and servers (server image uses a text/console installer) Bhyve support for TrueOS Server Install FreeBSD base is synced with 12.0-CURRENT as of December 4th, 2017 (Github commit: 209d01f) FreeBSD ports tree is synced as of November 30th (pre-FLAVOR changes) Lumina Desktop has been updated/developed from 1.3.0 to 1.4.1 PCDM now supports multiple simultaneous graphical sessions Removable devices are now managed through the “automounter” service. Devices are “announced” as available to the system via *.desktop shortcuts in /media. These shortcuts also contain a variety of optional “Actions” that may be performed on the device. Devices are only mounted while they are being used (such as when browsing via the command line or a file manager). Devices are automatically unmounted as soon as they stop being accessed. Integrated support for all major filesystems (UFS, EXT, FAT, NTFS, ExFAT, etc..) NOTE: The Lumina desktop is the only one which supports this functionality at the present time. The TrueOS update system has moved to an “active” update backend. This means that the user will need to actually start the update process by clicking the “Update Now” button in SysAdm, Lumina, or PCDM (as well as the command-line option). The staging of the update files is still performed automatically by default but this (and many other options) can be easily changed in the “Update Manager” settings as desired. Known Errata: [VirtualBox] Running FreeBSD within a VirtualBox VM is known to occasionally receive non-existent mouse clicks – particularly when using a scroll wheel or two-finger scroll. Quick Links: TrueOS Forums TrueOS Bugs TrueOS Handbook TrueOS Community Chat on Telegram ***TrueOS Version 17.12 (2017, December) is now available for download from the TrueOS website. Both the STABLE and UNSTABLE package repositories have also been updated in-sync with each other, so current users only need to follow the prompts about updating their system to run the new release.
We are also pleased to announce the availability of TrueOS Sponsorships! If you would like to help contribute to the project financially we now have the ability to accept both one-time donations as well as recurring monthly donations which wil help us advocate for TrueOS around the world. Thank you all for using and supporting TrueOS!OpenBSD Workstation Guide
Design Goals User actions should complete instantaneously. While I understand if compiling code and rendering videos takes time, opening programs and moving windows should have no observable delay. The system should use minimalist tools. Corollary: cache data offline when possible. Everything from OpenStreetMaps to StackExchange can be stored locally. No reason to repeatedly hit the internet to query them. This also improves privacy because the initial download is indiscriminate and doesn’t reveal personal queries or patterns of computer activity. No idling program should use a perceptible amount of CPU. Why does CalendarAgent on my Macbook sometimes use 150% CPU for fifteen minutes? Who knows. Why are background ChromeHelpers chugging along at upper-single-digit CPU? I didn’t realize that holding a rendered DOM could be so challenging. Avoid interpreted languages, web-based desktop apps, and JavaScript garbage. There, I said it. Take your Electron apps with you to /dev/null! Stability. Old fashioned programs on a conservative OS on quality mainstream hardware. There are enough challenges to tackle without a bleeding edge system being one of them. Delegate to quality hardware components. Why use a janky ncurses software audio mixer when you can use…an actual audio mixer? Hardware privacy. No cameras or microphones that I can’t physically disconnect. Also real hardware protection for cryptographic keys. Software privacy. Commercial software and operating systems have gotten so terrible about this. I even catch Mac command line tools trying to call Google Analytics. Sorry homebrew, your cute emojis don’t make up for the surveillance. The Hardware Core
No operating system pre-loaded (Cut out the “Windows tax”) Intel Xeon E3-1270 v6 processor (Supports ECC ram) 16 GB (2x8 GB) DDR4-2400 ECC Unbuffered memory (2400Mhz is the full memory clock speed supported by the Xeon) 256 GB HP Z Turbo Drive G2 PCIe SSD (Uses NVMe rather than SATA for faster throughput, supported by nvme(4)) No graphics card (We’ll add our own)To get the best hardware for the money I’m opting for a desktop computer. Haven’t had one since the early 2000s and it feels anachronistic, but it will outperform a laptop of similar cost. After much searching, I found the HP Z240 Tower Workstation. It’s no-nonsense and supports exactly the customizations I was looking for:
Intel® Ethernet I210-T1 PCIe (Supported by em(4))
A modest discrete video card will enable 2D Glamor acceleration on X11. The Radeon HD 6450 (sold separately) is fanless and listed as supported by radeon(4).
Why build a solid computer and not protect it? Externally, the APC BR1300G UPS will protect the system from power surges and abrupt shutdowns.
Peripherals
Audio/VideoThe Matias Ergo Pro uses mechanical switches for that old fashioned clicky sound. It also includes dedicated buttons along the side for copying and pasting. Why is that cool? Well, it improves secondary selection, a technique that Sun computers used but time forgot.
Since we’re talking about a home office workstation, you may want a printer. The higher quality printers speak PostScript and PDF natively. Unix machines connect to them on TCP port 9100 and send PostScript commands directly. (You can print via telnet if you know the commands!) The Brother HL-L5100DN is a duplex LaserJet which allows that “raw” TCP printing.
BackupsI know a lot of people enjoy surrounding themselves with a wall of monitors like they’re in the heart of NASA Mission Control, but I find multi-monitor setups slightly disorienting. It introduces an extra bit of cognitive overhead to determine which monitor is for what exactly. That’s why I’d go with a modest, crisp Dell UltraSharp 24" U2417H. It’s 1080p and yeah there are 4k monitors nowadays, but text and icons are small enough as it is for me!
If I ever considered a second monitor it would be e-ink for comfortably reading electronic copies of books or long articles. The price is currently too high to justify the purchase, but the most promising monitor seems to be the Dasung Paperlike.
In the other direction, video input, it’s more flexible to use a general-purpose HDMI capture box like the Rongyuxuan than settle on a particular webcam. This allows hooking up a real camera, or any other video device.
Although the motherboard for this system has built-in audio, we should use a card with better OpenBSD support. The WBTUO PCIe card uses a C-Media CMI8768 chipset, handled by cmpci(4). The card provides S/PDIFF in and out ports if you ever want to use an external DAC or ADC.
The way to connect it with other things is with a dedicated hardware mixer. The Behringer Xenyx 802 has all the connections needed, and the ability to route audio to and from the computer and a variety of devices at once. The mixer may seem an odd peripheral, but I want to mix the computer with an old fashioned CD player, ham radio gear, and amplifier so this unifies the audio setup.
When doing remote pair programming or video team meetings it’s nice to have a quality microphone. The best ones for this kind of work are directional, with a cardioid reception pattern. The MXL 770 condenser mic is perfect, and uses a powered XLR connection supplied by the mixer.
CryptographyWe’re going dead simple and old-school, back to tapes. There are a set of tape standards called LTO-n. As n increases the tape capacity gets bigger, but the tape drive gets more expensive. In my opinion the best balance these days for the home user is LTO-3. You can usually find an HP Ultrium 960 LTO-3 on eBay for 150 dollars. The cartridges hold 800GB and are about 15 dollars apiece. Hard drives keep coming down in price, but these tapes are very cheap and simpler than keeping a bunch of disk drives. Also tape has proven longevity, and good recoverability.
To use old fashioned tech like this you need a SCSI host bus adapter like the Adaptec 29320LPE, supported by ahd(4).
The SoftwareYou don’t want to generate and store secret keys on a general purpose network attached computer. The attack surface is a mile wide. Generating or manipulating “offline” secret keys needs to happen on a separate computer with no network access.
Little boards like the Raspberry Pi would be good except they use ARM processors (incompatible with Tails OS) and have wifi. The JaguarBoard is a small x86 machine with no wireless capability. Just switch the keyboard and monitor over to this machine for your “cleanroom.”
jaguar board: Generating keys requires entropy. The Linux kernel on Tails samples system properties to generate randomness, but why not help it out with a dedicated true random number generator (TRNG)? Bit Babbler supplies pure randomness at a high bitrate through USB. (OneRNG works better on the OpenBSD main system, via uonerng(4).)
bit babbler: This little computer will save its results onto a OpenPGP Smartcard V2.1. This card provides write-only access to keys, and computes cryptographic primitives internally to sign and encrypt messages. To use it with a regular computer, hook up a Cherry ST2000 card reader. This reader has a PIN pad built in, so no keylogger on the main computer could even obtain your decryption PIN.We take the beefed up hardware above and pair it with ninja-fast software written in C. Some text-based, others raw X11 graphical apps unencumbered by ties to any specific window manager.
I’d advise OpenBSD for the underlying operating system, not a Linux. OpenBSD has greater internal consistency, their man pages are impeccable, and they make it a priority to prune old code to keep the system minimal.What Have We Learned from the PDP-11?
“Right from the get go, Bell is letting us know that the success of any computer project is not abstractly building the best computer but building the right computer, and that takes context.”The paper I have chosen tonight is a retrospective on a computer design. It is one of a series of papers by Gordon Bell, and various co-authors, spanning the design, growth, and eventual replacement of the companies iconic line of PDP-11 mini computers.
This year represents the 60th anniversary of the founding of the company that produced the PDP-11. It is also 40 years since this paper was written, so I thought it would be entertaining to review Bell’s retrospective through the lens of our own 20/20 hindsight.
To set the scene for this paper, first we should talk a little about the company that produced the PDP-11, the Digital Equipment Corporation of Maynard, Massachusetts. Better known as DEC.
It’s also worth noting that the name PDP is an acronym for “Programmed Data Processor”, as at the time, computers had a reputation of being large, complicated, and expensive machines, and DEC’s venture capitalists would not support them if they built a “computer”
A computer is not solely determined by its architecture; it reflects the technological, economic, and human aspects of the environment in which it was designed and built. […] The finished computer is a product of the total design environment.
“It’s also interesting to note Bell’s concern that additional registers would confuse the user. In the early 1970’s the assumption that the machine would be programmed directly in assembly was still the prevailing mindset.”It is the nature of computer engineering to be goal-oriented, with pressure to produce deliverable products. It is therefore difficult to plan for an extensive lifetime.
Because of the open nature of the PDP-11, anything which interpreted the instructions according to the processor specification, was a PDP-11, so there had been a rush within DEC, once it was clear that the PDP-11 market was heating up, to build implementations; you had different groups building fast, expensive ones and cost reduced slower ones
The first weakness of minicomputers was their limited addressing capability. The biggest (and most common) mistake that can be made in a computer design is that of not providing enough address bits for memory addressing and management.
A second weakness of minicomputers was their tendency not to have enough registers. This was corrected for the PDP-11 by providing eight 16-bit registers. Later, six 32-bit registers were added for floating-point arithmetic. […] More registers would increase the multiprogramming context switch time and confuse the user.
“Nowadays it’s hard to imagine hardware that doesn’t have a notion of a stack, but consider that a stack isn’t important if you don’t need recursion.” “The design for the PDP-11 was laid down in 1969 and if we look at the programming languages of the time, FORTRAN and COBOL, neither supported recursive function calls. The function call sequence would often store the return address at a blank word at the start of the procedure making recursion impossible.”A third weakness of minicomputers was their lack of hardware stack capability. In the PDP-11, this was solved with the autoincrement/autodecrement addressing mechanism. This solution is unique to the PDP-11 and has proven to be exceptionally useful. (In fact, it has been copied by other designers.)
“Strings and character handling were of increasing importance during the 1960’s as scientific and business computing converged. The predominant character encodings at the time were 6 bit character sets which provided just enough space for upper case letters, the digits 0 to 9, space, and a few punctuation characters sufficient for printing financial reports.” “Because memory was so expensive, placing one 6 bit character into a 12 or 18 bit word was simply unacceptable so characters would be packed into words. This proved efficient for storage, but complex for operations like move, compare, and concatenate, which had to account for a character appearing in the top or bottom of the word, expending valuable words of program storage to cope.” “The problem was addressed in the PDP-11 by allowing the machine to operate on memory as both a 16-bit word, and the increasingly popular 8-bit byte. The expenditure of 2 additional bits per character was felt to be worth it for simpler string handling, and also eased the adoption of the increasingly popular 7-bit ASCII standard of which DEC were a proponent at the time. Bell concludes this point with the throw away line:”A fourth weakness, limited interrupt capability and slow context switching, was essentially solved with the device of UNIBUS interrupt vectors, which direct device interrupts.
The basic mechanism is very fast, requiring only four memory cycles from the time an interrupt request is issued until the first instruction of the interrupt routine begins execution.
A fifth weakness of prior minicomputers, inadequate character-handling capability, was met in the PDP-11 by providing direct byte addressing capability.
The article, and the papers it is summarizing, contain a lot more than we could possibly dig into even if we dedicated the entire show to the topic ***Although string instructions are not yet provided in the hardware, the common string operations (move, compare, concatenate) can be programmed with very short loops.
A sixth weakness, the inability to use read-only memories, was avoided in the PDP-11. Most code written for the PDP-11 tends to be pure and reentrant without special effort by the programmer, allowing a read-only memory (ROM) to be used directly.
A seventh weakness, one common to many minicomputers, was primitive I/O capabilities.
A ninth weakness of minicomputers was the high cost of programming them. Many users program in assembly language, without the comfortable environment of editors, file systems, and debuggers available on bigger systems. The PDP-11 does not seem to have overcome this weakness, although it appears that more complex systems are being built successfully with the PDP-11 than with its predecessors, the PDP-8 and PDP-15.
The problems faced by computer designers can usually be attributed to one of two causes: inexperience or second-systemitis
Before the PDP-11, there was no UNIX. Before the PDP-11, there was no C, this is the computer that C was designed on. If you want to know why the classical C int is 16 bits wide, it’s because of the PDP-11. UNIX bought us ideas such as pipes, everything is a file, and interactive computing.
UNIX, which had arrived at Berkley in 1974 aboard a tape carried by Ken Thompson, would evolve into the west coast flavoured Berkley Systems Distribution. Berkeley UNIX had been ported to the VAX by the start of the 1980’s and was thriving as the counter cultural alternative to DEC’s own VMS operating system. Berkeley UNIX spawned a new generation of hackers who would go on to form companies like Sun micro systems, and languages like Self, which lead directly to the development of Java. UNIX was ported to a bewildering array of computer systems during the 80’s and the fallout from the UNIX wars gave us the various BSD operating systems who continue to this day.News Roundup
Two-factor authentication SSH with Duo in FreeBSD 11
This setup uses an SSH key as the first factor of authentication. Please watch Part 1 on setting up SSH keys and how to scp it to your server.
Register for a free account at Duo.com
Install the Duo package on your FreeBSD server
Log into the Duo site > Applications > Protect an Application > Search for Unix application > Protect this Application This will generate the keys we need to configure Duo. Edit the Duo config file using the course notes templatepkg install -y duo
Example configvi /usr/local/etc/pam_duo.conf
Change the permissions of the Duo config file. If the[duo] ; Duo integration key ikey = Integration key goes here ; Duo secret key skey = Secret key goes here ; Duo API host host = API hostname goes hereWed, 20 Dec 2017 - 1h 47min - 824 - 224: The Bus Factor
We try to answer what happens to an open source project after a developers death, we tell you about the last bootstrapped tech company in Silicon Valley, we have an update to the NetBSD Thread sanitizer, and show how to use use cabal on OpenBSD
This episode was brought to you by
Headlines
Life after death, for code
The Bus FactorYOU'VE PROBABLY NEVER heard of the late Jim Weirich or his software. But you've almost certainly used apps built on his work.
Weirich helped create several key tools for Ruby, the popular programming language used to write the code for sites like Hulu, Kickstarter, Twitter, and countless others. His code was open source, meaning that anyone could use it and modify it. "He was a seminal member of the western world's Ruby community," says Justin Searls, a Ruby developer and co-founder of the software company Test Double.
When Weirich died in 2014, Searls noticed that no one was maintaining one of Weirich's software-testing tools. That meant there would be no one to approve changes if other developers submitted bug fixes, security patches, or other improvements. Any tests that relied on the tool would eventually fail, as the code became outdated and incompatible with newer tech.
The incident highlights a growing concern in the open-source software community. What happens to code after programmers pass away? Much has been written about what happens to social-media accounts after users die. But its been less of an issue among programmers. In part, thats because most companies and governments relied on commercial software maintained by teams of people. But today, more programs rely on obscure but crucial software like Weirich's.
Some open-source projects are well known, such as the Linux operating system or Google's artificial-intelligence framework TensorFlow. But each of these projects depend on smaller libraries of open-source code. And those libraries depend on other libraries. The result is a complex, but largely hidden, web of software dependencies.
That can create big problems, as in 2014 when a security vulnerability known as "Heartbleed" was found in OpenSSL, an open-source program used by nearly every website that processes credit- or debit-card payments. The software comes bundled with most versions of Linux, but was maintained by a small team of volunteers who didn't have the time or resources to do extensive security audits. Shortly after the Heartbleed fiasco, a security issue was discovered in another common open-source application called Bash that left countless web servers and other devices vulnerable to attack.
There are surely more undiscovered vulnerabilities. Libraries.io, a group that analyzes connections between software projects, has identified more than 2,400 open-source libraries that are used in at least 1,000 other programs but have received little attention from the open-source community.
Security problems are only one part of the issue. If software libraries aren't kept up to date, they may stop working with newer software. That means an application that depends on an outdated library may not work after a user updates other software. When a developer dies or abandons a project, everyone who depends on that software can be affected. Last year when programmer Azer Koçulu deleted a tiny library called Leftpad from the internet, it created ripple effects that reportedly caused headaches at Facebook, Netflix, and elsewhere.
A 'Dead-Man's Switch'The fewer people with ownership of a piece of software, the greater the risk that it could be orphaned. Developers even have a morbid name for this: the bus factor, meaning the number of people who would have to be hit by a bus before there's no one left to maintain the project. Libraries.io has identified about 3,000 open-source libraries that are used in many other programs but have only a handful of contributors.
Orphaned projects are a risk of using open-source software, though commercial software makers can leave users in a similar bind when they stop supporting or updating older programs. In some cases, motivated programmers adopt orphaned open-source code.
That's what Searls did with one of Weirichs projects. Weirich's most-popular projects had co-managers by the time of his death. But Searls noticed one, the testing tool Rspec-Given, hadn't been handed off, and wanted to take responsibility for updating it. But he ran into a few snags along the way.
Rspec-Given's code was hosted on the popular code-hosting and collaboration site GitHub, home to 67 million codebases. Weirich's Rspec-Given page on GitHub was the main place for people to report bugs or to volunteer to help improve the code. But GitHub wouldnt give Searls control of the page, because Weirich had not named him before he died. So Searls had to create a new copy of the code, and host it elsewhere. He also had to convince the operators of Ruby Gems, a package-management system for distributing code, to use his version of Rspec-Given, instead of Weirich's, so that all users would have access to Searls changes. GitHub declined to discuss its policies around transferring control of projects.That solved potential problems related to Rspec-Given, but it opened Searls' eyes to the many things that could go wrong. Its easy to see open source as a purely technical phenomenon, Searls says. But once something takes off and is depended on by hundreds of other people, it becomes a social phenomenon as well.
The maintainers of most package-management systems have at least an ad-hoc process for transferring control over a library, but that process usually depends on someone noticing that a project has been orphaned and then volunteering to adopt it. "We dont have an official policy mostly because it hasnt come up all that often," says Evan Phoenix of the Ruby Gems project. "We do have an adviser council that is used to decide these types of things case by case."
Some package managers now monitor their libraries and flag widely used projects that haven't been updated in a long time. Neil Bowers, who helps maintain a package manager for the programming language Perl, says he sometimes seeks out volunteers to take over orphan projects. Bowers says his group vets claims that a project has been abandoned, and the people proposing to take it over.Taking over Rspec-Given inspired Searls, who was only 30 at the time, to make a will and a succession plan for his own open-source projects. There are other things developers can do to help future-proof their work. They can, for example, transfer the copyrights to a foundation, such as the Apache Foundation. But many open-source projects essentially start as hobbies, so programmers may not think to transfer ownership until it is too late.
Searls suggests that GitHub and package managers such as Gems could add something like a "dead man's switch" to their platform, which would allow programmers to automatically transfer ownership of a project or an account to someone else if the creator doesnt log in or make changes after a set period of time.
But a transition plan means more than just giving people access to the code. Michael Droettboom, who took over a popular mathematics library called Matplotlib after its creator John Hunter died in 2012, points out that successors also need to understand the code. "Sometimes there are parts of the code that only one person understands," he says. "The knowledge exists only in one person's head."
That means getting people involved in a project earlier, ideally as soon as it is used by people other than the original developer. That has another advantage, Searls points out, in distributing the work of maintaining a project to help prevent developer burnout.The Last Bootstrapped Tech Company In Silicon Valley
Live By The VC, Die By The VCMy business partner, Matt Olander, and I were intimately familiar with the ups and downs of the Silicon Valley tech industry when we acquired the remnants of our then-employer BSDis enterprise computer business in 2002 and assumed the roles of CEO and CTO. Fast-forward to today, and we still work in the same buildings where BSDi started in 1996, though youd hardly recognize them today.
As the business grew from a startup to a global brand, our success came from always ensuring we ran a profitable business. While that may sound obvious, keep in mind that we are in the heart of Silicon Valley where venture capitalists hunt for the unicorn company that will skyrocket to a billion-dollar valuation. Unicorns like Facebook and Twitter unquestionably exist, but they are the exception.
Grow Up, But On Your Own TermsAfter careful consideration, Matt and I decided to bootstrap our company rather than seek funding. The first dot-com bubble had recently burst, and we were seeing close friends lose their jobs right and left at VC-funded companies based on dubious business plans. While we did not have much cash on hand, we did have a customer base and treasured those customers as our greatest asset. We concluded that meeting their needs was the surest path to meeting ours, and the rest would simply be details to address individually. This strategy ended up working so well that we have many of the same customers to this day.
After deciding to bootstrap, we made a decision on a matter that has left egg on the face of many of our competitors: We seated sales next to support under one roof at our manufacturing facility in Silicon Valley. Dell's decision to outsource some of its support overseas in the early 2000s was the greatest gift it could have given us. Some of our sales and support staff have worked with the same clients for over a decade, and we concluded that no amount of funding could buy that mutual loyalty. While accepting venture capital or an acquisition may make you rich, it does not guarantee that your customers, employees or even business will be taken care of. Our motto is, Treat your customers like friends and employees like family, and we have an incredibly low employee turnover to show for it.
Thanks to these principles, iXsystems has remained employee-owned, debt-free and profitable from the day we took it over -- all without VC funding, which is why we call ourselves the "last bootstrapped tech company in Silicon Valley." As a result, we now provide enterprise servers to thousands of customers, including top Fortune 500 companies, research and educational institutions, all branches of the military, and numerous government entities.
Over time, however, we realized that we were selling more and more third-party data storage systems with every order. We saw this as a new opportunity. We had partnered with several storage vendors to meet our customers needs, but every time we did, we opened a can of worms with regard to supporting our customers to our standards. Given a choice of risking being dragged down by our partners or outmaneuvered by competitors with their own storage portfolios, we made a conscious decision to develop a line of storage products that would not only complement our enterprise servers but tightly integrate with them.
To accelerate this effort, we adopted the FreeNAS open-source software-defined storage project in 2009 and havent looked back. The move enabled us to focus on storage, fully leveraging our experience with enterprise hardware and our open source heritage in equal measures. We saw many storage startups appear every quarter, struggling to establish their niche in a sea of competitors. We wondered how theyd instantly master hardware to avoid the partnering mistakes that we made years ago, given that storage hardware and software are truly inseparable at the enterprise level. We entered the storage market with the required hardware expertise, capacity and, most importantly, revenue, allowing us to develop our storage line at our own pace.
Relate your experiences to each fad: Our industry is a firehose of fads and buzzwords, and it can be difficult to distinguish the genuine trends from the flops. Analyze every new buzzword in terms of your own products, services and experiences, and monitor customer trends even more carefully. Some buzzwords will even formalize things you have been doing for years. Value personal relationships: Companies come and go, but you will maintain many clients and colleagues for decades, regardless of the hat they currently wear. Encourage relationship building at every level of your company because you may encounter someone again. Trust your instincts and your colleagues: No contractual terms or credit rating system can beat the instincts you will develop over time for judging the ability of individuals and companies to deliver. You know your business, employees and customers best.By not having the external pressure from VCs or shareholders that your competitors have, you're free to set your own priorities and charge fair prices for your products. Our customers consistently tell us how refreshing our sales and marketing approaches are. We consider honesty, transparency and responsible marketing the only viable strategy when youre bootstrapped. Your reputation with your customers and vendors should mean everything to you, and we can honestly say that the loyalty we have developed is priceless.
So how can your startup venture down a similar path? Here's our advice for playing the long game:Looking back, I don't think Id change a thing. We need to be in Silicon Valley for the prime customers, vendors and talent, and its a point of pride that our customers recognize how different we are from the norm. Free of a venture capital runway and driven by these principles, we look forward to the next 20 years in this highly-competitive industry.
Creating an AS for fun and profit
To become your own Internet Service Provider with customers who pay you to access the Internet, or be your own web hosting provider with customers who pay you to be accessible from the Internet, or your own transit provider who has customers who pay you to move their customer's packets to other people's customers, you need a few things: Your own public IP address space allocated to you by an Internet numbering organization Your own Autonomous System Number (ASN) to identify your network as separate from everyone else's networks At least one router connected to a different autonomous system speaking the Border Gateway Protocol to tell the rest of the Internet that your address space is accessible from your autonomous system.At its core, the Internet is an interconnected fabric of separate networks. Each network which makes up the Internet is operated independently and only interconnects with other networks in clearly defined places.
For smaller networks like your home, the interaction between your network and the rest of the Internet is usually pretty simple: you buy an Internet service plan from an ISP (Internet Service Provider), they give you some kind of hand-off through something like a DSL or cable modem, and give you access to "the entire Internet". Your router (which is likely also a WiFi access point and Ethernet switch) then only needs to know about two things; your local computers and devices are on one side, and the ENTIRE Internet is on the other side of that network link given to you by your ISP.
For most people, that's the extent of what's needed to be understood about how the Internet works. Pick the best ISP, buy a connection from them, and attach computers needing access to the Internet. And that's fine, as long as you're happy with only having one Internet connection from one vendor, who will lend you some arbitrary IP address(es) for the extend of your service agreement, but that starts not being good enough when you don't want to be beholden to a single ISP or a single connection for your connectivity to the Internet.
That also isn't good enough if you are an Internet Service Provider so you are literally a part of the Internet. You can't assume that the entire Internet is that way when half of the Internet is actually in the other direction. This is when you really have to start thinking about the Internet and treating the Internet as a very large mesh of independent connected organizations instead of an abstract cloud icon on the edge of your local network map. Which is pretty much never for most of us.
Almost no one needs to consider the Internet at this level. The long flight of steps from DSL for your apartment up to needing to be an integral part of the Internet means that pretty much regardless of what level of Internet service you need for your projects, you can probably pay someone else to provide it and don't need to sit down and learn how BGP works and what an Autonomous System is. But let's ignore that for one second, and talk about how to become your own ISP.
My motivation was twofold: One of my friends and I sat down and figured it out that splitting the cost of a rack in Hurricane Electric's FMT2 data center marginally lowered our monthly hosting expenses vs all the paid services we're using scattered across the Internet which can all be condensed into this one rack. And this first reason on its own is a perfectly valid justification for paying for co-location space at a data center like Hurricane Electric's, but isn't actually a valid reason for running it as an autonomous system, because Hurricane Electric will gladly let you use their address space for your servers hosted in their building. That's usually part of the deal when you pay for space in a data center: power, cooling, Internet connectivity, and your own IP addresses. Another one of my friends challenged me to do it as an Autonomous System. So admittedly, my justification for going through the additional trouble to set up this single rack of servers as an AS is a little more tenuous. I will readily admit that, more than anything else, this was a "hold my beer" sort of engineering moment, and not something that is at all needed to achieve what we actually needed (a rack to park all our servers in).So... I recently set up my own autonomous system... and I don't really have a fantastic justification for it...
So here's how I set up a multi-homed autonomous system on a shoe-string budget: Step 1. Found a Company Step 2. Get Yourself Public Address Space Step 3. Find Yourself Multiple Other Autonomous Systems to Peer With Step 4. Apply for an Autonomous System Number Step 5. Source a Router Capable of Handling the Entire Internet Routing Table Step 6. Turn it All On and Pray And we're off to the races.But what the hell; I've figured out how to do it, so I figured it would make an entertaining blog post.
At this point, Hurricane Electric is feeding us all ~700k routes for the Internet, we're feeding them our two routes for our local IPv4 and IPv6 subnets, and all that's left to do is order all our cross-connects to other ASes in the building willing to peer with us (mostly for fun) and load in all our servers to build our own personal corner of the Internet.
The only major goof so far has been accidentally feeding the full IPv6 table to our first other peer that we turned on, but thankfully he has a much more powerful supervisor than the Sup720-BXL, so he just sent me an email to knock that off, a little fiddling with my BGP egress policies, and we were all set.
In the end, setting up my own autonomous system wasn't exactly simple, it was definitely not justified, but some times in life you just need to take the more difficult path. And there's a certain amount of pride in being able to claim that I'm part of the actual Internet. That's pretty neat.
And of course, thanks to all of my friends who variously contributed parts, pieces, resources, and know-how to this on-going project. I had to pull in a lot of favors to pull this off, and I appreciate it.News Roundup
One year checkpoint and Thread Sanitizer update
The past year has been started
Wed, 13 Dec 2017 - 1h 40min - 823 - 223: Compile once, debug twice
Picking a compiler for debuggability, how to port Rust apps to FreeBSD, what the point of Docker is on FreeBSD/Solaris, another EuroBSDcon recap, and network manager control in OpenBSD
This episode was brought to you by
Headlines
Compile once, Debug twice: Picking a compiler for debuggability, part 1 of 3
An interesting look into why when you try to debug a crash, you can often find all of the useful information has been optimized out
In another exampleHave you ever had an assert get triggered only to result in a useless core dump with missing variable information or an invalid callstack?
Common factors that go into selecting a C or C++ compiler are: availability, correctness, compilation speed and application performance. A factor that is often neglected is debug information quality, which symbolic debuggers use to reconcile application executable state to the source-code form that is familiar to most software engineers.
When production builds of an application fail, the level of access to program state directly impacts the ability for a software engineer to investigate and fix a bug. If a compiler has optimized out a variable or is unable to express to a symbolic debugger how to reconstruct the value of a variable, the engineers investigation process is significantly impacted. Either the engineer has to attempt to recreate the problem, iterate through speculative fixes or attempt to perform prohibitively expensive debugging, such as reconstructing program state through executable code analysis.
Debug information quality is in fact not proportionally related to the quality of the generated executable code and wildly varies from compiler to compiler.
Different compilers emit debug information at varying levels of quality and accuracy. However, certain optimizations will certainly impact any debuggers ability to generate accurate stack traces or extract variable values.
In the above program, the value of argv is extracted and then the program is paused. The ck_pr_load_ptr function performs a read from the region of memory pointed to by argv, in a manner that prevents the compiler from performing optimization on it. This ensures that the memory access occurs and for this reason, the value of argv must be accessible by the time ck_pr_load_ptr is executed.
When compiled with gcc, the debugger fails to find the value of the variable. The compiler determines that the value of argv is no longer needed after the ck_pr_load_ptr operation and so doesnt bother paying the cost of saving the value.
Some optimizations generate executable code whose call stack cannot be sufficiently disambiguated to reconcile a call stack that mirrors that of the source program. Two common culprits for this are tail call optimization and basic block commoning.
So the for loop, changes into a while loop, with a condition of the end of the inputIf the program receives a first argument of 1, then function is called with the argument of "a". If the program receives a first argument of 2, then function is called with the argument of "b". However, if we compile this program with clang, the stack traces in both cases are identical! clang informs the debugger that the function f invoked the function("b") branch where x = 2 even if x = 1.
Though some optimizations will certainly impact the accuracy of a symbolic debugger, some compilers simply lack the ability to generate debug information in the presence of certain optimizations. One common optimization is induction variable elimination. A variable thats incremented or decremented by a constant on every iteration of a loop or derived from another variable that follows this pattern, is an induction variable.
Coupled with other optimizations, the compiler is then able to generate code that doesnt actually rely on a dedicated counter variable i for maintaining the current offset into buffer.
As you can see, i is completely optimized out. The compiler determines it doesnt have to pay the cost of maintaining the induction variable i. It maintains the pointer in the register %rdi. The code is effectively rewritten to something closer to this:
Looking forward to part 2 ***We have shown some common optimizations that may get in the way of the debuggability of your application and demonstrated a disparity in debug information quality across two popular compilers. In the next blog post of this series, we will examine how gcc and clang stack up with regards to debug information quality across a myriad of synthetic applications and real world applications.
This is how you can port your rust application to FreeBSD
This is how you can port your rust application to FreeBSDThe FreeBSD Ports Collection is the way almost everyone installs applications (ports) on FreeBSD. Like everything else about FreeBSD, it is primarily a volunteer effort. It is important to keep this in mind when reading this document.
In FreeBSD, anyone may submit a new port, or volunteer to maintain an existing unmaintained port. No special commit privilege is needed.
For this guide I will use fd tool written by David Peter as example project.Prerequisites
FreeBSD installation (VM is fine) Local ports tree (done via svn) portlint (located at devel/portlint) poudriere (located at ports-mgmt/poudriere)[optional]Getting ports tree
When you install FreeBSD opt-out of the ports tree. Install svn:
Poudrierepkg install svn svn checkout https://svn.freebsd.org/ports/head /usr/ports
MakefileSometimes you might get asked to show poudriere build log, sometimes you wont. Its good to have anyway. If you choose to use poudriere, use ZFS. There are plenty of guides on the subject. FreeBSD Porters Handbook is the most complete source of information on porting to FreeBSD.
Port metadataWhole porting process in most cases is writing one Makefile. I recommend doing something like this.
Here is the one I wrote for fd:Each port must have one primary category in case of fd it will be sysutils, therefore it's located in /usr/ports/systuils/fd.
PORTNAME= fd CATEGORIES= sysutilsSince this port conflicts with other util named fd I specified package suffix as: PKGNAMESUFFIX= -find and indicate conflict: CONFLICTS_INSTALL= fd-[0-9]*. That means to install it from packages user will have to type:
Licensespkg install fd-findThis section is different for every port, but in case of fd it's pretty straightforward:
LICENSE= MIT APACHE20 LICENSE_COMB= dualSince fd includes the text of licenses you should do this as well:
DistfilesLICENSE_FILE_MIT= ${WRKSRC}/LICENSE-MIT LICENSE_FILE_APACHE20= ${WRKSRC}/LICENSE-APACHEFreeBSD has a requirement that all ports must allow offline building. That means you have specified which files are needed to be downloaded. Luckily we now have helpers to download GitHub sources directly from GitHub:
USE_GITHUB= yes GH_ACCOUNT= sharkdpSince PORTNANE is fd it will try to download sources for sharkdp/fd. By default it's going to download tag:
${DISTVERSIONPREFIX}${DISTVERSION}${DISTVERSIONSUFFIX}
Extra packagesfd uses v as the prefix, therefore we need to specify: DISTVERSIONPREFIX= v.
It's also possible to specify GH_TAGNAME in case tag name doesn't match that pattern.There are very few rust projects that are standalone and use no crates dependencies. Its used to be PITA to make it work offline, but now cargo is a first class citizen in ports:
USES= cargo CARGO_CRATES= aho-corasick-0.6.3 \ atty-0.2.3 \ # and so goes onYes, you have to specify each dependency. Luckily, there is a magic awk script that turns Cargo.lock into what you need. Execute make cargo-crates in the port root. This will fail because you're missing checksum for the original source files:
make makesum make cargo-crates
CARGO_OUTThis will give you what you need. Double check that result is correct. There is a way to ignore checksum error, but I cant remember Execute make makesum again.
If. build.rs relies on that you have to change it. fd allows you to use SHELL_COMPLETIONS_DIR to specify where completions go, while ripgrep doesn't. In our case we just specify SHELL_COMPLETIONS_DIR:
PLISTSHELL_COMPLETIONS_DIR= ${WRKDIR}/shell-completions-dir CARGO_ENV= SHELL_COMPLETIONS_DIR=${SHELL_COMPLETIONS_DIR}FreeBSD is very strict about files its installing and it wont allow you to install random files that get lost. You have to specify which files youre installing. In this case, its just two:
PLIST_FILES= bin/fd \ man/man1/fd.1.gzNote that sources for fd have uncompressed man file, while here its listed as compressed. If port installs a lot of files, specify them in pkg-plist like here. To actually install them:
Shell completionspost-install: @${STRIP_CMD} ${STAGEDIR}${PREFIX}/bin/fd ${INSTALL_MAN}${WRKSRC}/doc/fd.1 ${STAGEDIR}${MAN1PREFIX}/man/man1clap-rs can generate shell completions for you, it's usually handled by build.rs script. First, we need to define options:
OPTIONS_DEFINE= BASH FISH ZSH # list options OPTIONS_DEFAULT= BASH FISH ZSH # select them by default BASH_PLIST_FILES= etc/bash_completion.d/fd.bash-completion FISH_PLIST_FILES= share/fish/completions/fd.fish ZSH_PLIST_FILES= share/zsh/site-functions/_fdTo actually install them:
Bonus round - Patching source codepost-install-BASH-on: @${MKDIR} ${STAGEDIR}${PREFIX}/etc/bash_completion.d ${INSTALL_DATA} ${SHELL_COMPLETIONS_DIR}/fd.bash-completion \ ${STAGEDIR}${PREFIX}/etc/bash_completion.d post-install-FISH-on: @${MKDIR} ${STAGEDIR}${PREFIX}/share/fish/completions ${INSTALL_DATA} ${SHELL_COMPLETIONS_DIR}/fd.fish \ ${STAGEDIR}${PREFIX}/share/fish/completions post-install-ZSH-on: @${MKDIR} ${STAGEDIR}${PREFIX}/share/zsh/site-functions ${INSTALL_DATA} ${SHELL_COMPLETIONS_DIR}/_fd \ ${STAGEDIR}${PREFIX}/share/zsh/site-functions
Go to work/ dir Copy file you want to patch and add .orig suffix to it Edit file you want to patchSometimes you have to patch it and send the patch upstream. Merging it upstream can take awhile, so you can patch it as part of the install process. An easy way to do it:
Execute make makepatch in port's root
Submitting port
First, make sure portlint -AC doesn't give you any errors or warnings. Second, make sure poudriere can build it on both amd64 and i386. If it can't??you have to either fix it or mark port broken for that arch.
Follow this steps like I did steps. If you have any issues you can always ask your question in freebsd-ports on freenode try to find your answer in porters handbook before asking.Conference Recap: EuroBSDCon 2017 Recap
The location was wonderful and I loved sneaking out and exploring the city when I could. From what I heard, it was the largest BSD conference in history, with over 320 attendees!
Each venue is unique and draws many local BSD enthusiasts, who normally wouldnt be able to travel to a conference. I love having the chance to talk to these people about how they are involved in the projects and what they would like to do. Most of the time, they are asking me questions about how they can get more involved and how we can help.
Magical is how I would describe the conference social event. To stand in front of the dinner cruise on the Seine, with the Eiffel Tower standing tall, lit up in the night, while working talking to our community members, was incredible. But, let me start at the beginning.
We attend these conferences to talk to our community members, to find out what they are working on, determine technologies that should be supported in FreeBSD, and what we can do to help and improve FreeBSD.
We started the week with a half-day board meeting on Wednesday. BSD conferences give us a chance to not only meet with community members around the world, but to have face-to-face meetings with our team members, who are also located around the world. We worked on refining our strategic direction and goals, determining what upcoming conferences we want FreeBSD presence at and who can give FreeBSD talks and workshops there, discussed current and potential software development projects, and discussed how we can help raise awareness about and increase the use of FreeBSD in Europe.
Thursday was the first day of the FreeBSD developer summit, led by our very own Benedict Reuschling. He surprised us all by having us participate in a very clever quiz on France. 45 of us signed into the software, where hed show the question on the screen and we had a limited amount of time to select our answers, with the results listed on the screen. It was actually a lot of fun, especially since they didnt publicize the names of the people who got the questions wrong. The lucky or most knowledgeable person on France, was des@freebsd.org.
Some of our board members ran tutorials in parallel to the summit. Kirk McKusick gave his legendary tutorial, An Introduction to the FreeBSD Open-Source Operating System , George Neville-Neil gave his tutorial, DTrace for Developers, and Benedict Reuschling gave a tutorial on, Managing BSD systems with Ansible.
I was pleased to have two chairs from ACM-W Europe run an Increasing Diversity in the BSDs BoF for the second year in a row. We broke up into three groups to discuss different gender bias situations, and what we can do to address these types of situations, to make the BSD projects more diverse, welcoming, and inclusive. At the end, people asked that we continue these discussions at future BSD conferences and suggested having an expert in the field give a talk on how to increase the diversity in our projects.
As I mentioned earlier, the social dinner was on a boat cruising along the Seine. I had a chance to talk to community members in a more social environment. With the conference being in France, we had a lot of first time attendees from France. I enjoyed talking to many of them, as well as other people I only get to see at the European conferences. Sunday was full of more presentations and conversations. During the closing session, I gave a short talk on the Foundation and the work we are doing. Then, Benedict Reuschling, Board Vice President, came up and gave out recognition awards to four FreeBSD contributors who have made an impact on the Project.News Roundup
Playing with the pine64
Daniel Jakots writes in his blog about his experiences with his two pine64 boards:
Getting OpenBSD on itFinding something to install on it
6 weeks ago, I ordered two pine64 units. I didn't (and still don't) have much plan for them, but I wanted to play with some cheap boards. I finally received them this week. Initially I wanted to install some Linux stuff on it, I didn't have much requirement so I thought I would just look what seems to be easy and/or the best supported systemd flavour. I headed over their wiki. Everything seems either not really maintained, done by some random people or both. I am not saying random people do bad things, just that installing some random things from the Internet is not really my cup of tea.
I heard about Armbian but the server flavour seems to be experimental so I got scared of it. And sadly, the whole things looks like to be alot undermanned.
So I went for OpenBSD because I know the stuff and who to harWkindly ask for help. Spoiler alert, it's boring because it just works.
That's it Then you can connect with the usualI downloaded miniroot62.fs, dd'ed it on the micro SD card. I was afraid I'd need to fiddle with some things like sysutils/dtb because I don't know what I would have needed to do. That's because I don't know what it does and for this precise reason I was wrong and I didn't need to do anything. So just dd the miniroot62.fs and you can go to next checkpoint.
I plugged an HDMI cable, ethernet cable and the power, it booted, I could read for 10 seconds but then it got dark. Of course it's because you need a serial console. Of course I didn't have one.
I thought about trying to install OpenBSD blindly, I could have probably succeeded with autoinstall buuuuuut
Following some good pieces of advice from OpenBSD people I bought some cp2102 (I didn't try to understand what it was or what were the other possibilities, I just wanted something that would work :D).
I looked how to plug the thing. It appears you can plug it on two different places but if you plug it on the Euler bus it could power a bit the board so if you try to reboot it, it would then mess with the power disruption and could lead a unclean reboot.
You just need to plug three cables: GND, TXD and RXD. Of course, the TXD goes on the RXD pin from the picture and the RXD goes on the TXD pin. Guess why I'm telling you that!$ cu -dl /dev/cuaU0 -s 115200
Whats the point of Docker on FreeBSD or Solaris?
Penguinisters are very keen on their docker, but for the rest of us it may be difficult to see what the fuss is all about its only been around a few years and everyones talking about it. And someone asked again today. What are we missing?
Well docker is a solution to a Linux (and Windows) problem that FreeBSD/Solaris doesnt have. Until recently, the Linux kernel only implemented the original user isolation model involving chroot. More recent kernels have had Control Groups added, which are intended to provide isolation for a group of processes (namespaces). This came out of Google, and theyve extended to concept to include processor resource allocation as one of the knobs, which could be a good idea for FreeBSD. The scheduler is aware of the JID of the process its about to schedule, and I might take a look in the forthcoming winter evenings. But I digress.
So if isolation (containerisation in Linux terms) is in the Linux kernel, what is Docker bringing to the party? The only thing I can think of is standardisation and an easy user interface (at the expense of having Python installed). You might think of it in similar terms to ezjail a complex system intended to do something that is otherwise very simple.
To make a jail in FreeBSD all you need do is copy the files for your system to a directory. This can even be a whole servers system disk if you like, and jails can run inside jails. You then create a very simple config file, giving the jail a name, the path to your files and an what IP addresses to pass through (if any) and youre done. Just type service jail nameofjal start, and off it goes.
Is there any advantage in running Docker? Well, in a way, there is. Docker has a repository of system images that you can just install and run, and this is what a lot of people want. Theyre a bit like virtual appliances, but not mind-numbingly inefficient.
You can actually run docker on FreeBSD. A port was done a couple of years ago, but it relies on the 64-bit Linux emulation that started to appear in 10.x. The newer the version of FreeBSD the better.
Docker is in ports/sysutils/docker-freebsd. It makes uses of jails instead of Linux cgroups, and requires ZFS rather than UFS for file system isolation. I believe the Linux version uses Union FS but I could be completely wrong on that.
The FreeBSD port works with the Docker hub repository, giving you access to thousands of pre-packaged system images to play with. And thats about as far as Ive ever tested it. If you want to run the really tricky stuff (like Windows) you probably want full hardware emulation and something like Xen. If you want to deploy or migrate FreeBSD or Solaris systems, just copy a new tarball in to the directory and go. Its a non-problem, so why make it more complicated?
Given the increasing frequency Docker turns up in conversations, its probably worth taking seriously as Linux applications get packaged up in to images for easy access. Jails/Zones may be more efficient, and Docker images are limited to binary, but convenience tends to win in many environments.Network Manager Control for OpenBSD
I pro
Wed, 06 Dec 2017 - 1h 51min - 822 - 222: How Netflix works
We take a look at two-faced Oracle, cover a FAMP installation, how Netflix works the complex stuff, and show you who the patron of yak shaving is.
This episode was brought to you by
Headlines
Why is Oracle so two-faced over open source?
Oracle loves open source.
I love you, you're perfect.Except when the database giant hates open source. Which, according to its recent lobbying of the US federal government, seems to be "most of the time". Yes, Oracle has recently joined the Cloud Native Computing Foundation (CNCF) to up its support for open-source Kubernetes and, yes, it has long supported (and contributed to) Linux. And, yes, Oracle has even gone so far as to (finally) open up Java development by putting it under a foundation's stewardship. Yet this same, seemingly open Oracle has actively hammered the US government to consider that "there is no math that can justify open source from a cost perspective as the cost of support plus the opportunity cost of forgoing features, functions, automation and security overwhelm any presumed cost savings." That punch to the face was delivered in a letter to Christopher Liddell, a former Microsoft CFO and now director of Trump's American Technology Council, by Kenneth Glueck, Oracle senior vice president.
The US government had courted input on its IT modernisation programme. Others writing back to Liddell included AT&T, Cisco, Microsoft and VMware.
In other words, based on its letter, what Oracle wants us to believe is that open source leads to greater costs and poorly secured, limply featured software. Nor is Oracle content to leave it there, also arguing that open source is exactly how the private sector does not function, seemingly forgetting that most of the leading infrastructure, big data, and mobile software today is open source.
Details! Rather than take this counterproductive detour into self-serving silliness, Oracle would do better to follow Microsoft's path. Microsoft, too, used to Janus-face its way through open source, simultaneously supporting and bashing it. Only under chief executive Satya Nadella's reign did Microsoft realise it's OK to fully embrace open source, and its financial results have loved the commitment. Oracle has much to learn, and emulate, in Microsoft's approach.
Lies, damned lies, and Oracle lobbyingNow change Oracle has never been particularly warm and fuzzy about open source. As founder Larry Ellison might put it, Oracle is a profit-seeking corporation, not a peace-loving charity. To the extent that Oracle embraces open source, therefore it does so for financial reward, just like every other corporation. Few, however, are as blunt as Oracle about this fact of corporate open-source life. As Ellison told the Financial Times back in 2006: "If an open-source product gets good enough, we'll simply take it. So the great thing about open source is nobody owns it a company like Oracle is free to take it for nothing, include it in our products and charge for support, and that's what we'll do. "So it is not disruptive at all you have to find places to add value. Once open source gets good enough, competing with it would be insane... We don't have to fight open source, we have to exploit open source." "Exploit" sounds about right. While Oracle doesn't crack the top-10 corporate contributors to the Linux kernel, it does register a respectable number 12, which helps it influence the platform enough to feel comfortable building its IaaS offering on Linux (and Xen for virtualisation). Oracle has also managed to continue growing MySQL's clout in the industry while improving it as a product and business. As for Kubernetes, Oracle's decision to join the CNCF also came with P&L strings attached. "CNCF technologies such as Kubernetes, Prometheus, gRPC and OpenTracing are critical parts of both our own and our customers' development toolchains," said Mark Cavage, vice president of software development at Oracle. One can argue that Oracle has figured out the exploitation angle reasonably well. This, however, refers to the right kind of exploitation, the kind that even free software activist Richard Stallman can love (or, at least, tolerate). But when it comes to government lobbying, Oracle looks a lot more like Mr Hyde than Dr Jekyll.
The Vendor Formerly Known As SatanThe current US president has many problems (OK, many, many problems), but his decision to follow the Obama administration's support for IT modernisation is commendable. Most recently, the Trump White House asked for feedback on how best to continue improving government IT. Oracle's response is high comedy in many respects. As TechDirt's Mike Masnick summarises, Oracle's "latest crusade is against open-source technology being used by the federal government and against the government hiring people out of Silicon Valley to help create more modern systems. Instead, Oracle would apparently prefer the government just give it lots of money." Oracle is very good at making lots of money. As such, its request for even more isn't too surprising. What is surprising is the brazenness of its position. As Masnick opines: "The sheer contempt found in Oracle's submission on IT modernization is pretty stunning." Why? Because Oracle contradicts much that it publicly states in other forums about open source and innovation. More than this, Oracle contradicts much of what we now know is essential to competitive differentiation in an increasingly software and data-driven world.
Take, for example, Oracle's contention that "significant IT development expertise is not... central to successful modernization efforts". What? In our "software is eating the world" existence Oracle clearly believes that CIOs are buyers, not doers: "The most important skill set of CIOs today is to critically compete and evaluate commercial alternatives to capture the benefits of innovation conducted at scale, and then to manage the implementation of those technologies efficiently." While there is some truth to Oracle's claim every project shouldn't be a custom one-off that must be supported forever it's crazy to think that a CIO government or otherwise is doing their job effectively by simply shovelling cash into vendors' bank accounts.
Indeed, as Masnick points out: "If it weren't for Oracle's failures, there might not even be a USDS [the US Digital Service created in 2014 to modernise federal IT]. USDS really grew out of the emergency hiring of some top-notch internet engineers in response to the Healthcare.gov rollout debacle. And if you don't recall, a big part of that debacle was blamed on Oracle's technology." In short, blindly giving money to Oracle and other big vendors is the opposite of IT modernisation.
In its letter to Liddell, Oracle proceeded to make the fantastic (by which I mean "silly and false") claim that "the fact is that the use of open-source software has been declining rapidly in the private sector". What?!? This is so incredibly untrue that Oracle should score points for being willing to say it out loud. Take a stroll through the most prominent software in big data (Hadoop, Spark, Kafka, etc.), mobile (Android), application development (Kubernetes, Docker), machine learning/AI (TensorFlow, MxNet), and compare it to Oracle's statement. One conclusion must be that Oracle believes its CIO audience is incredibly stupid. Oracle then tells a half-truth by declaring: "There is no math that can justify open source from a cost perspective." How so? Because "the cost of support plus the opportunity cost of forgoing features, functions, automation and security overwhelm any presumed cost savings." Which I guess is why Oracle doesn't use any open source like Linux, Kubernetes, etc. in its services.
Oops.The thing is, Oracle doesn't need to do this and, for its own good, shouldn't do this. After all, we already know how this plays out. We need only look at what happened with Microsoft. Remember when Microsoft wanted us to "get the facts" about Linux? Now it's a big-time contributor to Linux. Remember when it told us open source was anti-American and a cancer? Now it aggressively contributes to a huge variety of open-source projects, some of them homegrown in Redmond, and tells the world that "Microsoft loves open source." Of course, Microsoft loves open source for the same reason any corporation does: it drives revenue as developers look to build applications filled with open-source components on Azure. There's nothing wrong with that.
Would Microsoft prefer government IT to purchase SQL Server instead of open-source-licensed PostgreSQL? Sure. But look for a single line in its response to the Trump executive order that signals "open source is bad". You won't find it. Why? Because Microsoft understands that open source is a friend, not foe, and has learned how to monetise it. Microsoft, in short, is no longer conflicted about open source. It can compete at the product level while embracing open source at the project level, which helps fuel its overall product and business strategy. Oracle isn't there yet, and is still stuck where Microsoft was a decade ago.
It's time to grow up, Oracle. For a company that builds great software and understands that it increasingly needs to depend on open source to build that software, it's disingenuous at best to lobby the US government to put the freeze on open source. Oracle needs to learn from Microsoft, stop worrying and love the open-source bomb. It was a key ingredient in Microsoft's resurgence. Maybe it could help Oracle get a cloud clue, too.Install FAMP on FreeBSD
The acronym FAMP refers to a set of free open source applications which are commonly used in Web server environments called Apache, MySQL and PHP on the FreeBSD operating system, which provides a server stack that provides web services, database and PHP. Prerequisites sudo Installed and working - Please read Apache PHP5 or PHP7 MySQL or MariaDB Install your favorite editor, ours is vi
Install PortsNote: You don't need to upgrade FreeBSD but make sure all patches have been installed and your port tree is up-2-date if you plan to update by ports.
Install Apacheportsnap fetch
You must use sudo for each indivdual command during installations. Please see link above for installing sudo.
Searching Available Apache Versions to Install
pkg search apache
How To find Your Server's Public IP AddressTo install Apache 2.4 using pkg. The apache 2.4 user account managing Apache is www in FreeBSD.
pkg install apache24
Confirmation yes prompt and hit y for yes to install Apache 2.4 This installs Apache and its dependencies.
Enable Apache use sysrc to update services to be started at boot time, Command below adds "apache24_enable="YES" to the /etc/rc.conf file. For sysrc commands please read
sysrc apache24_enable=yes
Start Apache
service apache24 start
Visit web address by accessing your server's public IP address in your web browser
Install MySQLIf you do not know what your server's public IP address is, there are a number of ways that you can find it. Usually, this is the address you use to connect to your server through SSH.
ifconfig vtnet0 | grep "inet " | awk '{ print $2 }'
Now that you have the public IP address, you may use it in your web browser's address bar to access your web server.
Install PHP5 or PHP70Now that we have our web server up and running, it is time to install MySQL, the relational database management system. The MySQL server will organize and provide access to databases where our server can store information. Install MySQL 5.7 using pkg by typing
pkg install mysql57-server
Enter y at the confirmation prompt. This installs the MySQL server and client packages.
To enable MySQL server as a service, add mysql_enable="YES" to the /etc/rc.conf file. This sysrc command will do just that
sysrc mysql_enable=yes
Now start the MySQL server
service mysql-server start
Now run the security script that will remove some dangerous defaults and slightly restrict access to your database system.Answer all questions to secure your newly installed MySQL database. Enter current password for root (enter for none): [RETURN] Your database system is now set up and we can move on.
Note: In these instructions we are using php5.7 not php7.0. We will be coming out with php7.0 instructions with FPM.pkg search php70
Install PHP70 you would do the following by typing
pkg install php70-mysqli mod_php70
Install PHP Modules (Optional)PHP is the component of our setup that will process code to display dynamic content. It can run scripts, connect to MySQL databases to get information, and hand the processed content over to the web server to display. We're going to install the mod_php, php-mysql, and php-mysqli packages. To install PHP 5.7 with pkg, run this command
pkg install mod_php56 php56-mysql php56-mysqli
Copy sample PHP configuration file into place.
cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini
Regenerate the system's cached information about your installed executable filesBefore using PHP, you must configure it to work with Apache.
Optional Install ExampleTo enhance the functionality of PHP, we can optionally install some additional modules. To see the available options for PHP 5.6 modules and libraries, you can type this into your system
pkg search php56
Get more information about each module you can look at the long description of the package by typing
pkg search -f apache24
Test PHP Processingpkg install php56-calendar
Configure Apache to Use PHP Module
Open the Apache configuration file
vim /usr/local/etc/apache24/Includes/php.conf<IfModule dir_module> DirectoryIndex index.php index.html Next, we will configure Apache to process requested PHP files with the PHP processor. Add these lines to the end of the file: <FilesMatch "\.php$"> SetHandler application/x-httpd-php </FilesMatch> <FilesMatch "\.phps$"> SetHandler application/x-httpd-php-source </FilesMatch> </IfModule>Now restart Apache to put the changes into effect
service apache24 restart
Details on info.phpBy default, the DocumentRoot is set to /usr/local/www/apache24/data. We can create the info.php file under that location by typing
vim /usr/local/www/apache24/data/info.php
Add following line to info.php and save it.<?php phpinfo(); ?>info.php file gives you information about your server from the perspective of PHP. It' useful for debugging and to ensure that your settings are being applied correctly.
If this was successful, then your PHP is working as expected.
You probably want to remove info.php after testing because it could actually give information about your server to unauthorized users. Remove file by typing
rm /usr/local/www/apache24/data/info.php
Note: Make sure Apache / meaning the root of Apache is owned by user which should have been created during the Apache install is the owner of the /usr/local/www structure.
That explains FAMP on FreeBSD.IXsystems
IXsystems TrueNAS X10 Torture Test & Fail Over Systems In Action with the ZFS File SystemHow Netflix works: what happens every time you hit Play
Not long ago, House of Cards came back for the fifth season, finally ending a long wait for binge watchers across the world who are interested in an American politicians ruthless ascendance to presidency. For them, kicking off a marathon is as simple as reaching out for your device or remote, opening the Netflix app and hitting Play. Simple, fast and instantly gratifying. What isnt as simple is what goes into running Netflix, a service that streams around 250 million hours of video per day to around 98 million paying subscribers in 190 countries. At this scale, providing quality entertainment in a matter of a few seconds to every user is no joke. And as much as it means building top-notch infrastructure at a scale no other Internet service has done before, it also means that a lot of participants in the experience have to be negotiated with and kept satiated??from production companies supplying the content, to internet providers dealing with the network traffic Netflix brings upon them.
This is, in short and in the most layman terms, how Netflix works.
Let us just try to understand how Netflix is structured on the technological side with a simple example.
Netflix literally ushered in a revolution around ten years ago by rewriting the applications that run the entire service to fit into a microservices architecture??which means that each application, or microservices code and resources are its very own. It will not share any of it with any other app by nature. And when two applications do need to talk to each other, they use an application programming interface (API)??a tightly-controlled set of rules that both programs can handle. Developers can now make many changes, small or huge, to each application as long as they ensure that it plays well with the API. And since the one program knows the others API properly, no change will break the exchange of information.
Netflix estimates that it uses around 700 microservices to control each of the many parts of what makes up the entire Netflix service: one microservice stores what all shows you watched, one deducts the monthly fee from your credit card, one provides your device with the correct video files that it can play, one takes a look at your watching history and uses algorithms to guess a list of movies that you will like, and one will provide the names and images of these movies to be shown in a list on the main menu. And thats the tip of the iceberg. Netflix engineers can make changes to any part of the application and can introduce new changes rapidly while ensuring that nothing else in the entire service breaks down.
They made a courageous decision to get rid of maintaining their own servers and move all of their stuff to the cloud??i.e. run everything on the servers of someone else who dealt with maintaining the hardware while Netflix engineers wrote hundreds of programs and deployed it on the servers rapidly. The someone else they chose for their cloud-based infrastructure is Amazon Web Services (AWS).
Netflix works on thousands of devices, and each of them play a different format of video and sound files. Another set of AWS servers take this original film file, and convert it into hundreds of files, each meant to play the entire show or film on a particular type of device and a particular screen size or video quality. One file will work exclusively on the iPad, one on a full HD Android phone, one on a Sony TV that can play 4K video and Dolby sound, one on a Windows computer, and so on. Even more of these files can be made with varying video qualities so that they are easier to load on a poor network connection. This is a process known as transcoding. A special piece of code is also added to these files to lock them with what is called digital rights management or DRM??a technological measure which prevents piracy of films.
The Netflix app or website determines what particular device you are using to watch, and fetches the exact file for that show meant to specially play on your particular device, with a particular video quality based on how fast your internet is at that moment.
Here, instead of relying on AWS servers, they install their very own around the world. But it has only one purpose??to store content smartly and deliver it to users. Netflix strikes deals with internet service providers and provides them the red box you saw above at no cost. ISPs install these along with their servers. These Open Connect boxes download the Netflix library for their region from the main servers in the US??if there are multiple of them, each will rather store content that is more popular with Netflix users in a region to prioritise speed. So a rarely watched film might take time to load more than a Stranger Things episode. Now, wWed, 29 Nov 2017 - 2h 07min - 821 - 221: BSD in Taiwan
Allan reports on his trip to BSD Taiwan, new versions of Lumina and GhostBSD are here, a bunch of OpenBSD p2k17 hackathon reports.
This episode was brought to you by
Headlines
Allans Trip Report from BSD Taiwan
BSD TW and Taiwan in general was a fun and interesting experience I arrived Thursday night and took the high speed train to Taipei main station, and then got on the Red line subway to the venue. The dorm rooms were on par with BSDCan, except the mattress was better. I spent Friday with a number of other FreeBSD developers doing touristy things. We went to Taipei 101, the worlds tallest building from 2004 - 2010. It also features the worlds fastest elevator (2004 - 2016), traveling at 60.6 km/h and transporting passengers from the 5th to 89th floor in 37 seconds. We also got to see the tuned mass damper, a 660 tonne steel pendulum suspended between the 92nd and 87th floors. This device resists the swaying of the building caused by high winds. There are interesting videos on display beside the damper, of its reaction during recent typhoons and earthquakes. The Taipei 101 building sits just 200 meters from a major fault line. Then we had excellent dumplings for lunch After walking around the city for a few more hours, we retired to a pub to escape the heat of the sunny Friday afternoon. Then came the best part of each day in Taipei, dinner! We continued our efforts to cause a nation wide shortage of dumplings Special thanks to Scott Tsai who took detailed notes for each of the presentations Saturday marked the start of the conference: Arun Thomas provided background and then a rundown of what is happening with the RISC-V architecture. Notes George Neville-Neil talked about using DTrace in distributed systems as an in-depth auditing system (who did what to whom and when). Notes Baptiste Daroussin presented Poudrière image, an extension of everyones favourite package building system, to build custom images of FreeBSD. There was discussion of making this generate ZFS based images as well, making it mesh very well with my talk the next day. Notes Brooks Davis presented his work on an API design for a replacement for mmap. It started with a history of address space management in the BSD family of operating systems going all the way back to the beginning. This overview of the feature and how it evolved filled in many gaps for me, and showed why the newer work would be beneficial. The motivation for the work includes further extensions to support the CHERI hardware platform. Notes Johannes M Dieterich gave an interesting presentation about using FreeBSD and GPU acceleration for high performance computing. One of the slides showed that amd64 has taken almost the entire market for the top 500 super computers, and that linux dominates the list, with only a few remaining non-linux systems. Sadly, at the supercomputing conference the next week, it was announced that linux has achieved 100% saturation of the top 500 super computers list. Johannes detailed the available tools, what ports are missing, what changes should be made to the base system (mostly OpenMP), and generally what FreeBSD needs to do to become a player in the supercomputer OS market. Johannes perspective is interesting, as he is a computational chemist, not a computer scientist. Those interested in improving the numerical libraries and GPU acceleration frameworks on FreeBSD should join the ports team. Notes The final talk of the day was Peter Grehan, who spoke about how graphics support in bhyve came to be. He provided a history of how the feature evolved, and where it stands today. Notes Afterwards, we traveled as a group to a large restaurant for dinner. There was even Mongolian Vodka, provided by Ganbold Tsagaankhuu of the FreeBSD project. Sunday: The first talk of the day Sunday was mine. I presented ZFS: Advanced Integration, mostly talking about how boot environments work, and the new libbe and be(1) tools that my GSoC student Kyle Kneitinger created to manage them. I talked about how they can be used for laptop and developer systems, but also how boot environments can be used to replace nanobsd for appliances (as already done in FreeNAS and pfSense). I also presented about zfsbootcfg (zfs nextboot), and some future extensions to it to make it even more useful in appliance type workloads. I also provided a rundown of new developments out of the ZFS developer summit, two weeks previous. Notes Theo de Raadt presented Mitigations and other real Security Features, and made his case for changing to a fail closed mode of interoperability. Computers cannot actually self heal, so lets stop pretending that they can. Notes Ruslan Bukin talked about doing the port of FreeBSD for RISC-V and writing the Device Drivers. Ruslan walked through the process step by step, leading members of the audience to suggest he turn it into a developers handbook article, explaining how to do the initial bringup on new hardware. Ruslan also showed off a FreeBSD/MIPS board he designed himself and had manufactured in China. Notes Mariusz Zaborski presented Case studies on sandboxing the base system with Capsicum. He discussed the challenges encountered as existing programs are modified to sandbox them, and recent advancements in the debugging tools available during that process. Mariusz also discussed the Casper service at length, including the features that are planned for 2018 and onwards. Notes The final presentation of the day was Mark Johnston on Memory Management Improvements in FreeBSD 12.0. This talk provided a very nice overview of the memory management system in FreeBSD, and then detailed some of the recent improvements. Notes The conference wrapped up with the Work-in-Progress session, including updates on: multi-device-at-once GELI attach, MP-safe networking on NetBSD, pkgsrc, NetBSD in general, BSD on Microsoft Azure, Mothra (send-pr for bugzilla), BSDMizer a machine learning compiler optimizer, Hyperledger Sawtooth (blockchain), and finally VIMAGE and pf testing on FreeBSD. Notes Group Photo BSDTW was a great conference. They are still considering if it should be an annual thing, trade off every 2nd year with AsiaBSDCon, or something else. In order to continue, BSD Taiwan requires more organizers and volunteers. They have regular meetups in Taipei if you are interested in getting involved. ***Lumina 1.4.0 released
The Lumina Theme Engine (and associated configuration utility) The Lumina theme engine is a new component of the core desktop, and provides enhanced theming capabilities for the desktop as well as all Qt5 applications. While it started out life as a fork of the qt5ct utility, it quickly grew all sorts of new features and functionality such as system-defined color profiles, modular theme components, and built-in editors/creators for all components. The backend of this engine is a standardized theme plugin for the Qt5 toolkit, so that all Qt5 applications will now present a unified appearance (if the application does not enforce a specific appearance/theme of its own). Users of the Lumina desktop will automatically have this plugin enabled: no special action is required. Please note that the older desktop theme system for Lumina has been rendered obsolete by the new engine, but a settings-conversion path has already been implemented which should transition your current settings to the new engine the first time you login to Lumina 1.4.0. Custom themes for the older system may not be converted though, but it is trivial to copy/paste any custom stylesheets from the old system into the editor for the new theme engine to register/re-apply them as desired. Lumina-Themes RepositoryI also want to give a shout-out to the trueos/lumina-themes github repository contributors. All of the wallpapers in the 1.4.0 screenshots I posted come from that package, and they are working on making more wallpapers, color palettes, and desktop styles for use with the Lumina Theme Engine. If your operating system does not currently provide a package for lumina-themes, I highly recommend that you make one as soon as possible!
The Lumina PDF Viewer (lumina-pdf)
This is a new, stand-alone desktop utility for viewing/printing/presenting PDF documents. It uses the poppler-qt5 library in the backend for rendering the document, but uses multi-threading in many ways (such as to speed up the loading of pages) to give the user a nice, streamlined utility for viewing PDF documents. There is also built-in presentation functionality which allows users to easily cast the document to a separate screen without mucking about in system menus or configuration utilities.Lumina PDF Viewer (1.4.0)
Important Packaging Changes
Other Changes of Note (in no particular order) lumina-config: Add a new page for changing audio theme (login, logout, low battery) Add option to replace fluxbox with some other WM (with appropriate warnings) Have the themes page redirect to launching the Lumina theme engine configuration utility. start-lumina-desktop: Auto-detect the active X11 displays and create a new display for the Lumina session (prevent conflict with prior graphical sessions). Add a process-failure counter & restart mechanism. This is particularly useful for restarting Fluxbox from time to time (such as after any monitor addition/removal) lumina-xconfig: Restart fluxbox after making any monitor changes with xrandr. This ensures a more reliable session. Implement a new 2D monitor layout mechanism. This allows for the placement of monitors anywhere in the X/Y plane, with simplification buttons for auto-tiling the monitors in each dimension based on their current location. Add the ability to save/load monitor profiles. Distinguish between the default monitor arrangement and the current monitor arrangement. Allow the user to set the current arrangement as the new default. lumina-desktop: Completely revamp the icon loading mechanisms so it should auto-update when the theme changes. Speed up the initialization of the desktop quite a bit. Prevent loading/probing files in the /net/ path for existence (assume they exist in the interest of providing shortcuts). On FreeBSD, these are special paths that actually pause the calling process in order to mount/load a network share before resuming the process, and can cause significant hangs in the desktop process. Add the ability to take a directory as a target for the wallpaper. This will open/probe the directory for any existing image files that it can use as a wallpaper and randomly select one. Remove the popup dialog prompting about system updates, and replace it with new Restart (with updates) buttons on the appropriate menus/windows instead. If no wallpapers selection is provided, try to use the lumina-nature wallpaper directory as the default, otherwise fall back on the original default wallpaper if the lumina-themes package is not installed. lumina-open: Make the *.desktop parsing a bit more flexible regarding quoted strings where there should not be any. If selecting which application to use, only overwrite the user-default app if the option is explicitly selected. lumina-fileinfo: Significant cleanup of this utility. Now it can be reliably used for creating/registering XDG application shortcuts. Add a whole host of new ZFS integrations: If a ZFS dataset is being examined, show all the ZFS properties for that dataset. If the file being examined exists within ZFS snapshots, show all the snapshots of the file lumina-fm: Significant use of additional multi-threading. Makes the loading of directories much faster (particularly ones with image files which need thumbnails) Add detection/warning when running as root user. Also add an option to launch a new instance of lumina-fm as the root user. [FreeBSD/TrueOS] Fix up the detection of the External Devices list to also list available devices for the autofs system. Fix up some drag and drop functionality. Expose the creation, extraction, and insertion of files into archives (requires lumina-archiver at runtime) Expand the Open With option into a menu of application suggestions in addition to the Other option which runs lumina-open to find an application. Provide an option to set the desktop wallpaper to the selected image file(s). (If the running desktop session is Lumina). lumina-mediaplayer: Enable the ability to playback local video files. (NOTE: If Qt5 is set to use the gstreamer multimedia backend, make sure you have the GL plugin installed for smooth video playback). lumina-archiver: Add CLI flags for auto-archive and auto-extract. This allows for programmatic/scriptable interactions with archives. That is not mentioning all of the little bugfixes, performance tweaks, and more that are also included in this release. ***One significant change of note for people who are packaging Lumina for their particular operating system is that the minimum supported versions of Qt for Lumina have been changed with this release:
lumina-core: Qt 5.4+ lumina-mediaplayer: Qt 5.7+ Everything else: Qt 5.2+Of course, using the latest version of the Qt5 libraries is always recommended.
The new lumina-pdf utility requires the availability of the poppler-qt5 library. The includes for this library on Ubuntu 17.10 were found to be installed outside of the normal include directories, so a special rule for it was added to our OS-Detect file in the Lumina source tree. If your particular operating system also places the the poppler include files in a non-standard place, please patch that file or send us the information and we can add more special rules for your particular OS.
When packaging for Linux distributions, the theme engine also requires the availability of some of the -dev packages for Qt itself when compiling the theme plugin. For additional information (specifically regarding Ubuntu builds), please take a look at a recent ticket on the Lumina repository.The strongest KASLR, ever?
Re: amd64: kernel aslr support
Initial designSo, I did it. Now the kernel sections are split in sub-blocks, and are all randomized independently. See my drawing [1]. What it means in practice, is that Kernel ASLR is much more difficult to defeat: a cache attack will at most allow you to know that a given range is mapped as executable for example, but you don't know which sub-block of .text it is; a kernel pointer leak will at most allow you to reconstruct the layout of one sub-block, but you don't know the layout and address of the remaining blocks, and there can be many.
The size and number of these blocks is controlled by the split-by-file parameter in Makefile.amd64. Right now it is set to 2MB, which produces a kernel with ~23 allocatable (ie useful at runtime) sections, which is a third of the total number supported (BTSPACE_NSEGS = 64). I will probably reduce this parameter a bit in the future, to 1.5MB, or even 1MB.
All of that leaves us with about the most advanced KASLR implementation available out there. There are ways to improve it even more, but you'll have to wait a few weeks for that.
If you want to try it out you need to make sure you have the latest versions of GENERIC_KASLR / prekern / bootloader. The instructions are still here, and haven't changed.
New designAs I said in the previous episode, I added in October a Kernel ASLR implementation in NetBSD for 64bit x86 CPUs. This implementation would randomize the location of the kernel in virtual memory as one block: a random VA would be chosen, and the kernel ELF sections would be mapped contiguously starting from there.
This design had several drawbacks: one leak, or one successful cache attack, could be enough to reconstruct the layout of the entire kernel and defeat KASLR.
NetBSDs new KASLR design significantly improves this situation.
Sub-blocksIn the new design, each kernel ELF section is randomized independently. That is to say, the base addresses of .text, .rodata, .data and .bss are not correlated. KASLR is already at this stage more difficult to defeat, since you would need a leak or cache attack on each of the kernel sections in order to reconstruct the in-memory kernel layout.
Then, starting from there, several techniques are used to strengthen the implementation even more.
Higher entropyThe kernel ELF sections are themselves split in sub-blocks of approximately 1MB. The kernel therefore goes from having:
{ .text .rodata .data .bss }to having
{ .text .text.0 .text.1 ... .text.i .rodata .rodata.0 ... .rodata.j ... .data ...etc }As of today, this produces a kernel with ~33 sections, each of which is mapped at a random address and in a random order.
This implies that there can be dozens of .text segments. Therefore, even if you are able to conduct a cache attack and determine that a given range of memory is mapped as executable, you dont know which sub-block of .text it is. If you manage to obtain a kernel pointer via a leak, you can at most guess the address of the section it finds itself in, but you dont know the layout of the remaining 32 sections. In other words, defeating this KASLR implementation is much more complicated than in the initial design.
Large pagesEach section is put in a 2MB-sized physical memory chunk. Given that the sections are 1MB in size, this leaves half of the 2MB chunk unused. Once in control, the prekern shifts the section within the chunk using a random offset, aligned to the ELF alignment constraint. This offset has a maximum value of 1MB, so that once shifted the section still resides in its initial 2MB chunk:
The prekern then maps these 2MB physical chunks at random virtual addresses; but addresses aligned to 2MB. For example, the two sections in Fig. A will be mapped at two distinct VAs:
There is a reason the sections are shifted in memory: it offers higher entropy. If we consider a .text.i section with a 64byte ELF alignment constraint, and give a look at the number of possibilities for the location of the section in memory:
The prekern shifts the 1MB section in its 2MB chunk, with an offset aligned to 64 bytes. So there are (2MB-1MB)/(64B)=214 possibilities for the offset.
Then, the prekern uses a 2MB-sized 2MB-aligned range of VA, chosen in a 2GB window. So there are (2GB-2MB)/(2MB)=210-1 possibilities for the VA.
Therefore, there are 214x(210-1)224 possible locations for the section. As a comparison with other systems:OS # of possibilities Linux 2^6 MacOS 2^8 Windows 2^13 NetBSD 2^24Of course, we are talking about one .text.i section here; the sections that will be mapped afterwards will have fewer location possibilities because some slots will be already occupied. However, this does not alter the fact that the resulting entropy is still higher than that of the other implementations. Note also that several sections have an alignment constraint smaller than 64 bytes, and that in such cases the entropy is even higher.
Countermeasures against TLB cache attacksThere is also a reason we chose to use 2MB-aligned 2MB-sized ranges of VAs: when the kernel is in control and initializes itself, it can now use large pages to map the physical 2MB chunks. This greatly improves memory access performance at the CPU level.
Strong?With the memory shift explained above, randomness is therefore enforced at both the physical and virtual levels: the address of the first page of a section does not equal the address of the section itself anymore.
It has, as a side effect, an interesting property: it can mostly mitigate TLB cache attacks. Such attacks operate at the virtual-page level; they will allow you to know that a given large page is mapped as executable, but you dont know where exactly within that page the section actually begins.This KASLR implementation, which splits the kernel in dozens of sub-blocks, randomizes them independently, while at the same time allowing for higher
Wed, 22 Nov 2017 - 1h 56min - 820 - 220: Opening ZFS in 2017
We have a first PS4 kernel exploit, the long awaited OpenZFS devsummit report by Allan, DragonflyBSD 5.0 is out, we show you vmadm to manage jails, and parallel processing with Unix tools.
This episode was brought to you by
Headlines
The First PS4 Kernel Exploit: Adieu
The First PS4 Kernel Exploit: Adieu
Vulnerability DiscoveryPlenty of time has passed since we first demonstrated Linux running on the PS4. Now we will step back a bit and explain how we managed to jump from the browser process into the kernel such that ps4-kexec et al. are usable. Over time, ps4 firmware revisions have progressively added many mitigations and in general tried to lock down the system. This post will mainly touch on vulnerabilities and issues which are not present on the latest releases, but should still be useful for people wanting to investigate ps4 security.
ExploitationAs previously explained, we were able to get a dump of the ps4 firmware 1.01 kernel via a PCIe man-in-the-middle attack. Like all FreeBSD kernels, this image included export symbols - symbols which are required to perform kernel and module initialization processes. However, the ps4 1.01 kernel also included full ELF symbols (obviously an oversight as they have been removed in later firmware versions). This oversight was beneficial to the reverse engineering process, although of course not a true prerequisite. Indeed, we began exploring the kernel by examining built-in metadata in the form of the syscall handler table - focusing on the ps4-specific entries.
Each process object in the kernel contains its own idt (ID Table) object. As can be inferred from the snippet above, the hash table essentially just stores pointers to opaque data blobs, along with a given kind and name. Entries may be accessed (and thus locked) with either read or write intent. Note that IDT_TYPE is not a bitfield consisting of only unique powers of 2. This means that if we can control the kind of an id_entry, we may be able to cause a type confusion to occur (it is assumed that we may control name).
AdieuTo an exploiter without ps4 background, it might seem that the easiest way to exploit this bug would be to take advantage of the write off the end of the mallocd namedobj_usr_t object. However, this turns out to be impossible (as far as I know) because of a side effect of the ps4 page size being changed to 0x4000 bytes (from the normal of 0x1000). It appears that in order to change the page size globally, the ps4 kernel developers opted to directly change the related macros. One of the many changes resulting from this is that the smallest actual amount of memory which malloc may give back to a caller becomes 0x40 bytes. While this also results in tons of memory being completely wasted, it does serve to nullify certain exploitation techniques (likely completely by accident ).
The namedobj exploit was present and exploitable (albeit using a slightly different method than described here) until it was fixed in firmware version 4.06. This vulnerability was also found and exploited by (at least) Chaitin Tech, so props to them! Taking a quick look at the 4.07 kernel, we can see a straightforward fix (4.06 is assumed to be identical - only had 4.07 on hand while writing this post):
int sys_namedobj_create(struct thread *td, void *args) { // ... rv = EINVAL; kind = *((_DWORD *)args + 4) if ( !(kind & 0x4000) && *(_QWORD *)args ) { // ... (unchanged) } return rv; }And so we say goodbye to a nice exploit. I hope you enjoyed this blast from the past :) Keep hacking!
OpenZFS Developer Summit 2017 Recap
OpenZFS Photo AlbumThe 5th annual OpenZFS Developer Summit was held in San Francisco on October 24-25. Hosted by Delphix at the Childrens Creativity Museum in San Francisco, over a hundred OpenZFS contributors from a wide variety of companies attended and collaborated during the conference and developer summit. iXsystems was a Gold sponsor and several iXsystems employees attended the conference, including the entire Technical Documentation Team, the Director of Engineering, the Senior Analyst, a Tier 3 Support Engineer, and a Tier 2 QA Engineer.
Day 1 of the conference had 9 highly detailed, informative, and interactive technical presentations from companies which use or contribute to OpenZFS. The presentations highlighted improvements to OpenZFS developed in-house at each of these companies, with most improvements looking to be made available to the entire OpenZFS community in the near to long term. Theres a lot of exciting stuff happening in the OpenZFS community and this post provides an overview of the presented features and proof-of-concepts.
The keynote was delivered by Mark Maybee who spoke about the past, present, and future of ZFS at Oracle. An original ZFS developer, he outlined the history of closed-source ZFS development after Oracles acquisition of Sun. ZFS has a fascinating history, as the project has evolved over the last decade in both open and closed source forms, independent of one another. While Oracles proprietary internal version of ZFS has diverged from OpenZFS, it has implemented many of the same features. Mark was very proud of the work his team had accomplished over the years, claiming Oracles ZFS products have accounted for over a billion dollars in sales and are used in the vast majority of Fortune 100 companies. However, with Oracle aggressively moving into cloud storage, the future of closed source ZFS is uncertain. Mark presented a few ideas to transform ZFS into a mainstream and standard file system, including adding more robust support for Linux.
Allan Jude from ScaleEngine talked about ZStandard, a new compression method he is developing in collaboration with Facebook. It offers compression comparable to gzip, but at speeds fast enough to keep up with hard drive bandwidth. According to early testing, it improves both the speed and compression efficiency over the current LZ4 compression algorithm. It also offers a new dictionary feature for improving image compression, which is of particular interest to Facebook. In addition, when using ZFS send and receive, it will adapt the compression ratio to make the most efficient use of the network bandwidth.
Currently, deleting a clone on ZFS is a time-consuming process, especially when dealing with large datasets that have diverged over time. Sara Hartse from Delphix described how clone fast delete speeds up clone deletion. Rather than traversing the entire dataset during clone deletion, changes to the clone are tracked in a live list which the delete process uses to determine which blocks to free. In addition, rather than having to wait for the clone to finish, the delete process backgrounds the task so you can keep working without any interruptions. Sara shared the findings of a test they ran on a clone with 500MB of data, which took 45 minutes to delete with the old method, and under a minute using the live list. This behavior is an optional property as it may not be appropriate for long-lived clones where deletion times are not a concern. At this time, it does not support promoted clones.
Olaf Faaland from Lawrence Livermore National Labs demonstrated the progress his team has made to improve ZFS pool imports with MMP (Multi-Modifier Protection), a watchdog system to make sure that ZFS pools in clustered High Availability environments are not imported by more than one host at a time. MMP uses uberblocks and other low-level ZFS features to monitor pool import status and otherwise safeguard the import process. MMP adds fields to on-disk metadata so it does not depend on hardware, such as SAS. It supports multi-node HA configs and does not affect non-HA systems. However, it does have issues with long I/O delays so existing HA software is recommended as an additional fallback.
Jörgen Lundman of GMO Internet gave an entertaining talk on the trials and tribulations of porting ZFS to OS X. As a bonus, he talked about porting ZFS to Windows, and showed a working demo. While not yet in a usable state, it demonstrated a proof-of-concept of ZFS support for other platforms.
Serapheim Dimitropoulos from Delphix discussed Faster Allocation with the Log Spacemap as a means of optimizing ZFS allocation performance. He began with an in-depth overview of metaslabs and how log spacemaps are used to track allocated and freed blocks. Since blocks are only allocated from loaded metaslabs but freed blocks may apply to any metaslab, over time logging the freed blocks to each appropriate metaslab with every txg becomes less efficient. Their solution is to create a pool-wide metaslab for unflushed entries.
Shailendra Tripathi from Tegile presented iFlash: Dynamic Adaptive L2ARC Caching. This was an interesting talk on what is required to allow very different classes of resources to share the same flash devicein their case, ZIL, L2ARC, and metadata. To achieve this, they needed to address the following differences for each class: queue priority, metaslab load policy, allocation, and data protection (as cache has no redundancy).
Isaac Huang of Intel introduced DRAID, or parity declustered RAID. Once available, this will provide the same levels of redundancy as traditional RAIDZ, providing the administrator doubles the amount of options for providing redundancy for their use case. The goals of DRAID are to address slow resilvering times and the write throughput of a single replacement drive being a bottleneck. This solution skips block pointer tree traversal when rebuilding the pool after drive failure, which is the cause of long resilver times. This means that redundancy is restored quickly, mitigating the risk of losing additional drives before the resilver completes, but it does require a scrub afterwards to confirm data integrity. This solution supports logical spares, which must be defined at vdev creation time, which are used to quickly restore the array.
Prakash Surya of Delphix described how ZIL commits currently occur in batches, where waiting threads have to wait for the batch to complete. His proposed solution was to replace batch commits and to instead notify the waiting thread after its ZIL commit in order to greatly increase throughput. A new tunable for the log write block timeout can also be used to log write blocks more efficiently.
Overall, the quality of the presentations at the 2017 OpenZFS conference was high. While quite technical, they clearly explained the scope of the problems being addressed and how the proposed solutions worked. We look forward to seeing the described features integrated into OpenZFS. The videos and slides for the presentations should be made available over the next month or so at the OpenZFS website.DragonflyBSD 5.0
DragonFly version 5.0 brings the first bootable release of HAMMER2, DragonFly's next generation file system. HAMMER2 Preliminary HAMMER2 support has been released into the wild as-of the 5.0 release. This support is considered EXPERIMENTAL and should generally not yet be used for production machines and important data. The boot loader will support both UFS and HAMMER2 /boot. The installer will still use a UFS /boot even for a HAMMER2 installation because the /boot partition is typically very small and HAMMER2, like HAMMER1, does not instantly free space when files are deleted or replaced. DragonFly 5.0 has single-image HAMMER2 support, with live dedup (for cp's), compression, fast recovery, snapshot, and boot support. HAMMER2 does not yet support multi-volume or clustering, though commands for it exist. Please use non-clustered single images for now. ipfw Updates IPFW has gone through a number of updates in DragonFly and now offers better performance. pf and ipfw3 are also still supported. Improved graphics support The i915 driver has been brought up to match what's in the Linux 4.7.10 kernel. Intel GPUs are supported up to the Kabylake generation. vga_switcheroo(4) module added, allowing the use of Intel GPUs on hybrid-graphics systems. The new apple_gmux driver enables switching to the Intel video chipset on dual Intel/NVIDIA and Intel/Radeon Macbook computers. Other user-affecting changes efisetup(8) added. DragonFly can now support over 900,000 processes on a single machine. Client-side SSH by default does not try password authentication, which is the default behavior in newer versions of OpenSSH. Pass an explicit '-o PasswordAuthentication=yes' or change /etc/ssh/ssh_config if you need the old behavior. Public key users are unaffected. Clang status A starting framework has been added for using clang as the alternate base compiler in DragonFly, to replace gcc 4.7. It's not yet complete. Clang can of course be added as a package. Package updates Many package updates but I think most notably we need to point to chrome60 finally getting into dports with accelerated video and graphics support. 64-bit status Note that DragonFly is a 64-bit-only operating system as of 4.6, and will not run on 32-bit hardware. AMD Ryzen is supported and DragonFly 5.0 has a workaround for a hardware bug. DragonFly quickly released a v5.0.1 with a few patches Download linkNews Roundup
(r)vmadm managing FreeBSD jails
Why another tool to manage jailsWe are releasing the first version (0.1.0) of our clone of vmadm for FreeBSD jails today. It is not done or feature complete, but it does provides basic functionality. At this point, we think it would be helpful to get it out there and get some feedback. As of today, it allows basic management of datasets, as well as creating, starting, stopping, and destroying jails.
What can we doHowever, before we go into details lets talk why we build yet another jail manager? It is not the frequent NIH syndrome, actually quite the opposite. In FiFo 0.9.2 we experimented with iocage as a way to control jails. While iocage is a useful tool when used as a CLI utility it has some issues when used programmatically.
When managing jails automatically and not via a CLI tool things like performance, or a machine parsable interface matter. While on a CLI it is acceptable if a call takes a second or two, for automatically consuming a tool this delay is problematic.
Another reason for the decision was that vmadm is an excellent tool. It is very well designed. SmartOs uses vmadm for years now. Given all that, we opted for adopting a proven interface rather than trying to create a new one. Since we already interface with it on SmartOS, we can reuse a majority of our management code between SmartOS and FreeBSD.
Whats nextToday we can manage datasets, which are jail templates in the form of ZFS volumes. We can list and serve them from a dataset-server, and fetch those we like want. At this point, we provide datasets for FreeBSD 10.0 to 11.1, but it is very likely that the list will grow. As an idea here is a community-driven list of datasets that exist for SmartOS today. Moreover, while those datasets will not work, we hope to see the same for BSD jails.
After fetching the dataset, we can define jails by using a JSON file. This file is compatible with the zone description used on SmartOS. It does not provide all the same features but a subset. Resources such as CPU and memory can be defined, networking configured, a dataset selected and necessary settings like hostname set.
With the jail created, vmadm allows managing its lifetime, starting, stopping it, accessing the console and finally destroying it. Updates to jails are supported to however as of today they are only taken into account after restarting the jail. However, this is in large parts not a technical impossibility but rather wasnt high up on the TODO list.
It is worth mentioning that vmadm will not pick up jails created in other tools or manually. Only using vmadm created jails was a conscious decision to prevent it interfering with existing setups or other utilities. While conventional tools can manage jails set up with vmadm just fine we use some special tricks like nested jails to allow for restrictions required for multi-tenancy that are hard or impossible to achieve otherwise.
You can get the tool from github. ***First and foremost we hope to get some feedback and perhaps community engagement. In the meantime, as announced earlier this year, we are hard at work integrating FreeBSD hypervisors in FiFo, and as of writing this, the core actions work quite well.
Right now only the barebone functions are supported, some of the output is not as clear as we would like. We hope to eventually add support for behyve to vmadm the same way that it supports KVM on SmartOS. Moreover, the groundwork for this already exists in the nested jail techniques we are using.
Other than that we are exploring ways to allow for PCI pass through in jails, something not possible in SmartOS zones right now that would be beneficial for some users.
In general, we want to improve compatibility with SmartOS as much as possible and features that we add over time should make the specifications invalid for SmartOS.Parallel processing with unix tools
There are various ways to use parallel processing in UNIX:
The example provided counts lines in parallel:piping
An often under appreciated idea in the unix pipe model is that the components of the pipe run in parallel. This is a key advantage leveraged when combining simple commands that do "one thing well"
split -n, xargs -P, parallel
Note programs that are invoked in parallel by these, need to output atomically for each item processed, which the GNU coreutils are careful to do for factor and sha*sum, etc. Generally commands that use stdio for output can be wrapped with thestdbuf -oLcommand to avoid intermixing lines from parallel invocations
make -j
Most implementations of make(1) now support the -j option to process targets in parallel. make(1) is generally a higher level tool designed to process disparate tasks and avoid reprocessing already generated targets. For example it is used very effictively when testing coreutils where about 700 tests can be processed in 13 seconds on a 40 core machine.
implicit threading
This goes against the unix model somewhat and definitely adds internal complexity to those tools. The advantages can be less data copying overhead, and simpler usage, though its use needs to be carefully considered. A disadvantage is that one loses the ability to easily distribute commands to separate systems. Examples are GNU sort(1) and turbo-linecount
You can either run multiple invocations of split in parallel on separate portions of the file like:The examples below will compare the above methods for implementing multi-processing, for the function of counting lines in a file. First of all let's generate some test data. We use both long and short lines to compare the overhead of the various methods compared to the core cost of the function being performed:
$ seq 100000000 > lines.txt # 100M lines
$ yes $(yes longline | head -n9) | head -n10000000 > long-lines.txt # 10M linesWe'll also define the add() { paste -d+ -s | bc; } helper function to add a list of numbers.
We'll use this command to count lines for most methods, so here is the base non multi-processing performance for comparison: $ time wc -l lines.txt $ time wc -l long-lines.txt split -n Note using -n alone is not enough to parallelize. For example this will run serially with each chunk, because since --filter may write files, the -n pertains to the number of files to split into rather than the number to process in parallel. $ time split -n$(nproc) --filter='wc -l' lines.txt | add
Note the following runs were done against cached files, and thus not I/O bound. Therefore we limit the number of processes in parallel to $(nproc), though you would generally benefit to raising that if your jobs are waiting on network or disk etc.$ time for i in $(seq $(nproc)); do
split -n$Wed, 15 Nov 2017 - 1h 54min
Podcasts similares a BSD Now
El Partidazo de COPE COPE- Herrera en COPE COPE
- La Linterna COPE
- Es la Mañana de Federico esRadio
- La noche de Cuesta esRadio
- Hondelatte Raconte Europe 1
- Au Coeur du Crime Europe1
- Affaires sensibles France Inter
- LEGEND Guillaume Pley
- El colegio invisible OndaCero
- La Rosa de los Vientos OndaCero
- Les grands dossiers de l'Histoire par Franck Ferrand Radio Classique
- Espacio en blanco Radio Nacional
- Enquêtes criminelles RTL
- Entrez dans l'Histoire RTL
- Le grand récit RTL
- Les Grosses Têtes RTL
- Les histoires incroyables de Pierre Bellemare RTL
- L'Heure Du Crime RTL
- El Larguero SER Podcast
- SER Historia SER Podcast
- Todo Concostrina SER Podcast
- Un Libro Una Hora SER Podcast
- HISTORIAS DE LA HISTORIA VIVA RADIO
Otros podcasts de Educación
- Japan Education Lawrence Denes
- 秋元才加とJOYのWeekly Japan!! TOKYO FM
- หลวงปู่ปราโมทย์ ปาโมชฺโช วัดสวนสันติธรรม dhamma.com
- The Ramsey Show Ramsey Network
- 台本なし英会話レッスン 英語のそーた & Reilly
- Eine Stunde History - Deutschlandfunk Nova Deutschlandfunk Nova
- Hapa英会話 Podcast Jun Senesac: バイリンガル 英会話 & ビジネス英語 講師
- Martha Debayle Martha Debayle
- Agora Historia Oficial Ágora Historia
- 武田鉄矢・今朝の三枚おろし 文化放送PodcastQR
- Tu Ingles! podcast Tu Ingles!
- 6 Minute English BBC Radio
- スピードラーニング ポッドキャスト[Speak UP Radio] 株式会社エスプリライン
- 김영철,피터빈트의 진짜 영국식 영어（김영철의 파워FM） SBS
- The Female Empowerment | Erfolgsgeschichten von Frauen The Female Empowerment | Erfolgsgeschichten von Frauen
- 일본어 무작정 따라하기 길벗 이지톡
- Historiepodden Acast
- VOA Learning English Podcast - VOA Learning English VOA Learning English
- 早安英文 早安英文
- The Money Case THE STANDARD
![スピードラーニング ポッドキャスト[Speak UP Radio]](https://is1-ssl.mzstatic.com/image/thumb/Podcasts125/v4/d1/ec/7d/d1ec7d38-310d-6195-c270-936e17bb5218/mza_5560451724791929410.jpg/300x300bb.jpg)
