NetBSD Planet


April 02, 2020

NetBSD Blog The GNU GDB Debugger and NetBSD (Part 1)
The NetBSD team of developers maintains two copies of GDB:

The process of maintaining a modern version (GPLv3) of GDB in basesystem is tainted with a constant extra cost. The NetBSD developers need to rebase the stack of local patches for the newer releases of the debugger and resurrect the support. The GDB project is under an active development and in active refactoring of the code, that was originally written in C, to C++.

Unfortunately we cannot abandon the local basesystem patches and rely on a pristine version as there is lack of feature parity in the pkgsrc version of GDB: no threading support, not operational support for most targets, no fork/vfork/etc events support, no auxv reading support on 64-bit kernels, no proper support of signals, single step etc.

Additionally there are extra GDB patches stored in pkgsrc-wip (created by me last year), that implement the gdbserver support for NetBSD/amd64. gdbserver is a GDB version that makes it possible to remotely debug other programs even across different Operating Systems and CPUs. This code has still not been merged into the mainline base-system version. This month, I have discovered that support needs to be reworked, as the preexisting source code directory hierarchy was rearranged.

Unless otherwise specified all the following changes were upstreamed to the mainstream GDB repository. According to the GDB schedule, the GDB10 branch point is planned on 2020-05-15 with release on 2020-06-05. It's a challenge to see how much the GDB support can be improved by then for NetBSD!

PSIM

The GDB debugger contains PSIM (Model of the PowerPC Architecture) originally developed by Andrew Cagney between 1994 and 1996. This is a simulator that contains, among other things, NetBSD support in the UEA mode. This means that GDB can run static programs prebuilt for NetBSD without execution on a real PowerPC hardware. In order to make it work, there is need to wrap the kernel interfaces such as syscalls, errno values and signals and handle them in the simulator.

I have updated the list of errno names and signal names with NetBSD 9.99.49.

It would be nice to still update the list of syscalls to reflect the current kernels, but I have deferred this into future.

bfd changes

The AArch64 (NetBSD/evbarm) target uses PT_GETREGS and PT_GETFPREGS operation names with the same Machine Dependent values as NetBSD/alpha and NetBSD/sparc. This knowledge is required as these values are used in core(5) files, as emitted by a crashing program. I've added a patch that recognizes these ELF notes in arm64 coredumps appropriately.

I've also added a new define constant NT_NETBSDCORE_AUXV. This allows properly identifying AUXV ELF notes in core files. Meanwhile I have implemented and added detection of LWPSTATUS notes. This note ships with meta information (name, signal context, TLS base, etc) about threads in a process in a core.

The number of ARM and MIPS boards supported by NetBSD is huge and there are multiple variations of them. I have fixed the detection macro in bfd to recognize more arm and mips NetBSD installations.

GDB/NetBSD fixes in CPU specific files

I have reached the state of GDB being more operational for more NetBSD ports out of the box. There were missing features and build issues that has been addressed. I have committed the following changes:

Now support for NetBSD in various CPU-specific files improved significantly, however there are still missing features, especially KGDB debugging and unwinding the stack over the signal trampoline. There are still smaller or larger changes that might be needed on per-port basis and I will keep working on them. There is need to develop at least proper aarch64 support as it is missing upstream. We might evaluate what to do with at least Itanium and RISCV.

CPU Generic improvements in the GDB codebase

I've switched the nbsd_nat_target::pid_to_exec_file() function from a logic of reading the /proc entries to a sysctl(3) based solution.

As the gdbserver support is around the corner, I have improved small parts of the code base to be compatibile with NetBSD. I've fixed the unconditional inclusion of alloca.h in gdbsupport. Another fix namespaced a local class reg, because it conflicted with the struct reg from the NetBSD headers.

The current logic of get_ptrace_pid function matches the semantics of other kernels suchs as Linux and FreeBSD. With the guidance of upstream developers, I have disabled this function completely for NetBSD instead of patching it for the NetBSD specific behavior of maintaining pairs PID+LWP for each internal ptid_t entry (that reflects the relation of PID, LWP and TID).

Plan for the next milestone

Finish reimplementing operational support of debugging of multi-threaded programs and upstream more patches, especially CPU-independent ones.

NetBSD Blog Extending support for the NetBSD-7 branch

Typically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7.

We've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong.

Security fixes will still be made to the NetBSD-7 branch.

We hope you're all safe. Stay home.

NetBSD Blog NetBSD 8.2 is available!

The third release in the NetBSD-8 is now available.

This release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability.

Some highlights include:

You can download binaries of NetBSD 8.2 from our Fastly-provided CDN.

For more details refer to the CHANGES-8.2 file.

Please note that we are looking for donations again, see Fundraising 2020.

Enjoy!

Maya

NetBSD.org NetBSD 8.2 released
NetBSD.org Extended support of the netbsd-7 branch

April 01, 2020

NetBSD.org New Developer in March 2020

March 20, 2020

Unix Stack Exchange How to change the font size in NetBSD (TTY no GUI)

NetBSD 7.1 (GENERIC.201703111743Z) amd64

In TTY without gui , the better screen resolution is set at boot time but the font is really to small.
So how to change that

On debian dpkg-reconfigure console-setup
in grub GRUB_GFXMODE=800x600

I find wsconscfg with a example :

wsconscfg -t 80x50 -e vt100 3

but I get

wsconscfg: WSDISPLAYIO_ADDSCREEN: Device not configured

I look at man wsconscfg wsconsctl wscons wsdisplay vga etc..
All of them link to the other but I can't find a way.


March 17, 2020

Super User Most Windows-like text editor that can be used through PuTTY in NetBSD?

Please help me find an editor that behaves more like the modern Windows editors do, that I could use on a NetBSD box through PuTTY.

This includes the absense of the distinction between view/insert/append modes, basic shortcuts such as Ctrl+Left/Right to jump a word at a time, and other niceties such as perhaps Home/End and PageUp/Down moving the cursor around (in my vi they just ding).

I can live without macros because I only ever tweak an occasional setting this way, and nothing else.

pico is pretty close, but I was wondering if there are other editors that might fit this bill?


March 16, 2020

DragonFly BSD Digest rcorder cross-pollination

rcorder-visualize.sh draws the dependencies for rc scripts using dot.  Originally in NetBSD, then in FreeBSD, now in DragonFly.


March 15, 2020

Unix Stack Exchange pkgin installation problem (NetBSD)

I just installed NetBSD 7.1.1 (i386) on my old laptop.

During the installation, I could not install pgkin (I don't know why), so I skipped it and now I have a NetBSD 7.1.1 installed on my laptop without pkgin.

My problem is "How to install pkgin on NetBSD (i386) ?"

I found this Unixmen tutorial and I followed it:

I tried :

#export PKG_PATH="http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/6.0_BETA_current/All/"
# pkg_add -v pkgin

And I got :

pkg_add: Can't process ftp://ftp.netbsd.org:80/pub/pkgsrc/packages/NetBSD/amd64/6.0_BETA_current/All/%0d/pkgin*: Not Found
pkg_add: no pkg found for 'pkgin',sorry.
pkg_add: 1 package addition failed

I know this is a wrong command because this ftp address is for amd64 while my laptop and this NetBSD is i386. (I can't find the correct command for i386 )

I also followed instructions of pkgin.net, and I did

git clone https://github.com/NetBSDfr/pkgin.git

on another computer and copied the output (which is a folder name pkgin) to my NetBSD (my NetBSD doesn't have 'git' command)

and then I did :

./configure --prefix=/usr/pkg --with-libraries=/usr/pkg/lib --with-includes=/usr/pkg/include

and then :

make

but I got :

#   compile  pkgin/summary.o
gcc -O2    -std=gnu99    -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-sign-compare  -Wno-traditional  -Wa,--fatal-warnings -Wreturn-type -Wswitch -Wshadow -Werror    -DPKGIN_VERSION=\""0.9.4 for NetBSD-7.1.1 i386"\" -DNETBSD  -g -DLOCALBASE=\"/usr/local\"           -DPKG_SYSCONFDIR=\"/usr/local/etc\"         -DPKG_DBDIR="\"/var/db/pkg\""           -DDEF_LOG_DIR="\"/var/db/pkg\""         -DPKGIN_DB=\"/var/db/pkgin\"            -DPKGTOOLS=\"/usr/local/sbin\" -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_LARGE_FILES -DCHECK_MACHINE_ARCH=\"i386\" -Iexternal -I. -I/usr/local/include  -c    summary.c
*** Error code 1

Stop.
make: stopped in /root/pkgin

I think this error occurs because of the dependencies. (which is mentioned in pkgin.net) but still, don't know how to install those dependencies.

EDIT: I found "http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/7.1.1/All/" but it still says

no pkg fond for 'pkgin', sorry

SOLVED:

** I solved the problem by writing 7.1 instead of 7.1.1**


March 14, 2020

DragonFly BSD Digest In Other BSDs for 2020/03/14

Throwing all the links I can out there.


March 10, 2020

NetBSD Blog Accomplishment of porting ptrace(2) test scenarios
This month I have finished porting ptrace(2) tests from other Operating Systems. I have determined which test scenarios were missing, compared to FreeBSD and Linux, and integrated them into the ATF framework. I have skipped some of the tests as the interesting behavior was already covered in existing tests (sometimes indirectly) or tools (like picotrace), or the NetBSD kernel exhibits different behavior.

As my work is reaching the end, I was trying to clean up the state with other projects.

ptrace(2) ATF tests

I have determined which test scenarios were missing and integrated them. Certain tests like wrapping FreeBSD specific pdfork(2) call were omitted as not applicable.

There are few new tests that are marked as expected failure for corner cases that are scheduled for fixing in future.

I have also worked on SIGCHLD-based debugging and analysis of its behavior. I have found out that SA_NOCLDWAIT behaves suspiciously. This flag passed to sigaction(2) is an extension. If set, the system will not create a zombie when the child exits, but the child process will be automatically waited for. The same effect can be achieved by setting the signal handler for SIGCHLD to SIG_IGN. Currently it behaves differently under a debugger as the child process is never collected and is waiting for parent to collect it. According to my research this behavior is unexpected. A potential fix might not be difficult in the kernel, but due to time constraints I have decided to add an ATF tests for this scenario, mark it as failed and include a comment deferring this case into future.

I have also refactored the remaining threaded tests, switching them from low-level LWP API to pthread(3) one.

Other changes

I was working on finishing projects that were left behind.

GDB and qemu upstreaming

I'm working on upstreaming NVMM support to mainline QEMU. This process is still ongoing.

I am slowly reducing the patchset against the GDB repository.

jemalloc changes

The jemalloc allocator is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support. It's the default allocator in the NetBSD Operating System since 2007.

There are a few workarounds that make jemalloc compatible with NetBSD internals and I was trying to remove them. Unfortunately, the allocator tries to initialize itself too early using a C++-like constructor and intercepts the first malloc(3). The is done before initializing libpthread, and the pthread startup code uses malloc() when registering pthread_atfork(3) callbacks. In order to make it work, we allow premature usage of the libpthread functionality. I was trying to correct this, but I've introduced slight regressions in corner cases. They are hard to debug as the allocator is corrupted internally and randomly misbehaves (hangs, occasional crashes). I've discussed with the upstream developers about addressing this properly, but as reproducing the setup needs familiarity with the process of development NetBSD, we are still working on it.

Meanwhile, I have managed to correct known Undefined Behavior issues in jemalloc and address all known issues working together with upstream.

syzkaller

I received write access to the syzkaller GitHub repository. I also helped to get Kernel MSan (unauthorized memory access) operational on the syzbot node.

Miscellaneous changes

I helped with the libc++ upgrade that was done by Michal Gorny (but still not merged into mainline). As part of this work we gained a support for errno codes for POSIX robust mutexes.

I have implemented missing DT_GNU_HASH support as specified by GNU and LLVM linkers. This code was based on the implementation from three other major BSDs.

The micro-UBSan implementation gained support for alignment_assumptions. A number of UBSan reports were addressed.

Plan for the next and the last milestone

Upstream gdbserver support and address as many remaining bugs as the time will permit.

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

http://netbsd.org/donations/#how-to-donate

NetBSD.org New Security Advisory: NetBSD-SA2020-002

March 09, 2020

NetBSD Blog Towards backtracing through signal trampolines and fresh libc++

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support, porting to i386.

During the last month, I've finally managed to create proper reproducers (and tests) for the remaining concurrent signal delivery problems. I have started working on backtracing through signal trampolines, and prepared a libc++ update.

NetBSD concurrent signal updates

While finishing the last report, I was trying to reproduce some of the concurrent test failures in LLDB with plain ptrace(). I've finally managed to do that and therefore discover the factor causing all my earlier attempts to fail — concurrent signal delivery works fine unless the signal is actually delivered to the process and handled by it.

Let me explain this a bit. When a signal is delivered to a debugged process (or one of its threads), it is stopped and the debugger receives stopping signal via waitpid(). Now, if the debugger wishes the signal to be delivered to the process (thread), it needs to pass the signal number as an argument to PT_CONTINUE. If it neglects to do so (passes 0), the signal is discarded.

My tests so far were doing precisely that — discarding the signal. However, once I modified them to pass it back, they started failing similarly to how LLDB tests are failing.

Whenever the debugged program receives concurrent signals to different threads and the debugger requests their delivery, the process is stopped with some of the signals multiple times. Curiously enough, during my testing every signal to a thread was reported at least once which means no signals were lost. I suspect that in an attempt to deliver pending concurrent signals the kernel is passing them again to the debugger rather than to the process itself.

I've used this research to extend testing of concurrent behavior. More specifically, I have:

  1. Made signal concurrency test into a reusable factory.

  2. Started testing passing signal back to the process.

  3. Extended the test to verify that signal is actually being delivered.

  4. Included catching newly-created processes in the test.

  5. Added concurrent breakpoints to the test.

  6. Added concurrent watchpoints to the test.

  7. Finally, started testing combination of simultaneous signals, breakpoints and watchpoints.

Research into backtrace through signal trampoline

The most important of the remaining tasks was to enhance LLDB with NetBSD signal trampoline support.

Signal trampolines on NetBSD

Signal trampolines are shortly covered by Signal delivery chapter of NetBSD Internals.

When a signal is delivered to a running program, the system needs to interrupt its execution and run its defined signal handler. Once the signal handler finishes, the program execution resumes where it left off. How this is achieved differs from system to system.

On NetBSD, so-called signal trampoline is used. The kernel (this is done by sendsig_siginfo() e.g. in amd64/machdep.c function on newer ABIs) saves the program context and executes the signal handler. When the signal handler returns, it returns to a trampoline function defined by the libc that restores the saved context and therefore resumes the program execution.

From debugger's perspective, the backtrace for a process interrupted in midst of a signal handler ends on this trampoline function. However, it is often considered useful to be able to know the status of the process just before the signal was received — and therefore, the point where program execution will continue. The goal in this point was to make LLDB aware of NetBSD's trampoline design and capable of locating and using the saved context to produce full backtrace.

The two possible solutions

There are two approaches to implementing signal trampoline handling:

  1. Explicitly detecting and processing signal trampolines in debugger.

  2. Adding CFI code to signal trampoline implementation in order to store the necessary information in libc itself.

GDB on NetBSD is currently using the first approach. The code (found in nbsd-tdep.c and e.g. amd64-nbsd-tdep.c) explicitly establishes whether the current frame corresponds to a signal trampoline, finds the saved context and processes it.

Long-term, the second approach is preferable. Instead of explicitly writing platform-specific code, we add CFI annotations to the trampoline code (e.g. in __sigtramp2.S). Those annotations are consumed by the toolchain and used to construct frame information inside the executable that can be afterwards consumed by the debugger.

Both approaches are therefore roughly equivalent. The main difference is that approach 1. stores platform-specific logic in the debugger, while approach 2. stores it in the executable for all debuggers to consume.

libc++ update

Another task to undergo during this period was to update libc++ in NetBSD src tree. It was last imported in 2015, to the version roughly corresponding to LLVM 3.7 release. This version is dated and has some bugs, particularly it is prone to miscompilation due to undefined behavior (e.g. segfault in std::map). I've decided to upgrade to the commit corresponding to the most recent LLVM/Clang update.

max_align_t visibility

The first problem I've hit after upgrading is that max_align_t is declared on NetBSD only for C11/C++11. However, on NetBSD libc++ is exposing it unconditionally.

Kamil Rytarowski proposed to expose max_align_t unconditionally in our headers as well. Joerg Sonnenberger on the other hand wants to change libc++ instead.

Missing errno constants

Another issue I've found is that NetBSD is missing the two errno constants for robust mutexes: EOWNERDEAD and ENOTRECOVERABLE. While libc++ has a hack to redefine them when missing, it seemed a better idea to assign them on our end.

I've learned that adding errno constants involves a few changes besides adding new constants:

  1. Adding mapping to Linux compat in sys/compat/linux/common/linux_errno.c.

  2. Adding descriptions to manpage lib/libc/sys/intro.2.

  3. Adding messages to libc catalogs.

  4. Enabling appropriate features in libstdc++.

  5. Adding new error codes to libdtrace.

  6. Adding errno mapping to NFS support in sys/nfs/nfs_subs.c.

While at it, I've made sure to make it harder to accidentally miss doing some of that in the future. Notably:

  1. I've added ATF tests to make sure that libc catalogs stay in sync with errno and signal descriptions in code.

  2. I've added a script to autogenerate libdtrace errno lists.

  3. I've added a compile-time assertion that NFS errno mapping covers all values.

The complete list of commits:

  1. Sync errno messages between catalog and errno.h

  2. Sync signal messages between catalog and sys_siglist

  3. Add tests for missing libc catalog entries

  4. PR standards/44921: Add errno consts for robust mutexes

  5. Enable EOWNERDEAD & ENOTRECOVERABLE in libstdc++

  6. Update dtrace errno.d mapping and add a script for it

  7. Update NFS errno mapping and add assert for correctness

The update

I have sent libc++ update to 01f3a59fb3e2542fce74c768718f594d0debd0da to the mailing list for review. The proposed patch set includes:

  1. Adjust the cleanup script for the new version.

  2. Cleaning up extraneous files from the old import (to make the diff clearer).

  3. Importing the new version and updating Makefiles.

  4. Moving headers to standard /usr/include/c++/v1 location for better interoperability.

  5. Moving libc++ to apache2 license group.

Future plans

This is the final month of my contract and therefore I would like to primarily focus on importing LLDB into src tree. As time permits, I will continue attempting to improve support for backtracing through signal trampolines.

The exact list of remaining tasks in my contract follows:

  1. Add support to backtrace through signal trampoline and extend the support to libexecinfo, unwind implementations (LLVM, nongnu). Examine adding CFI support to interfaces that need it to provide more stable backtraces (both kernel and userland).

  2. Add support for aarch64 target.

  3. Stabilize LLDB and address breaking tests from the test suite.

  4. Merge LLDB with the base system (under LLVM-style distribution).

This work is sponsored by The NetBSD Foundation

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

https://netbsd.org/donations/#how-to-donate


March 06, 2020

Stack Overflow How do I set root password for mysql 5.7 on netbsd

I'm trying to get mySQL 5.7 working on NetBSD (unix). When I try "mysql -u root" I get

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

When I try "mysqld --initialize-insecure" I get

[ERROR] --initialize specified but the data directory has files in it.

What do I do?


March 02, 2020

Emile Heitor Let's Encrypt certificates using LEGO

This post is more like a self-reminder on how I setup automatic SSL/TLS certificate renewal on my servers.

I chose LEGO to handle my certificates renewal with Let’s Encrypt because it’s simple to use, has no dependency, great documentation and is worked on at a constant pace.

I found this and this articles very useful, but they are outdated in their use of the tls and http parameters. So here are my notes.

This procedure is Debian GNU/Linux based but I also used it pretty much as-is on NetBSD and FreeBSD, only nginx related PATHs changed.

nginx

In /etc/nginx/sites-available/default:

1
include /etc/nginx/sites-available/letsencrypt;

Which contains:

1
2
3
4
location /.well-known/acme-challenge {
proxy_pass http://127.0.0.1:8181;
proxy_set_header Host $host;
}

Check nginx.conf syntax then reload it.

1
2
$ sudo nginx -t
$ sudo nginx -s reload

Let’s Encrypt

Go to a writable directory where LEGO will write the challenges

1
$ cd ~/www/letsencrypt

To request and install a new certificate, first prepare nginx.conf:

/etc/nginx/sites-available/https:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
listen443 ssl;
listen[::]:443 ssl;
server_namekorriban.imil.net;

ssl_certificate/usr/local/etc/letsencrypt/certificates/korriban.imil.net.crt;
ssl_certificate_key/usr/local/etc/letsencrypt/certificates/korriban.imil.net.key;

# ...

# for future use of the tls challenge
location /.well-known/acme-challenge {
proxy_pass http://127.0.0.1:4444;
proxy_set_header Host $host;
}

# ...

Then execute LEGO with desired parameters:

1
$ sudo lego --email="[email protected]" --domains="korriban.imil.net" --http.port :8181 --http --path=/usr/local/etc/letsencrypt run

And check / reload nginx.

If you need a multi-domain certificate, simply add multiples --domains.

To renew the certificate using the tls challenge if it expires within 30 days:

1
$ sudo /usr/local/bin/lego --email="[email protected]" --domains="korriban.imil.net" --tls.port :4444 --tls --path=/usr/local/etc/letsencrypt renew --days 30

Automatic renewal

1
$ cat bin/lerenew.sh
1
2
3
4
5
#!/bin/sh

cd /home/imil/www/letsencrypt
/usr/local/bin/lego --email="[email protected]" --domains="korriban.imil.net" --tls.port :4444 --tls --path=/usr/local/etc/letsencrypt renew --days 30
nginx -s reload

cron

1
00 3 * * * /home/imil/bin/lerenew.sh >/home/imil/log/lerenew.log 2>&1

reload mail system

My mail system also uses Let’s Encrypt certificates, it is hosted on virtual machines in the same subnet as the web server, which exports /usr/local/etc/letsencrypt/ via NFS. The mail server runs NetBSD. In order to watch any modifications in the certificates directory, I use direvent, which will call a script to reload both dovecot and postfix:

1
$ cat /usr/pkg/etc/direvent.conf
1
2
3
4
5
6
7
8
9
10
11
syslog {
facility local0;
tag "direvent";
print-priority yes;
}

watcher {
path /usr/local/etc/letsencrypt/certificates;
event write;
command "/home/imil/bin/mailrestart.sh";
}
1
$ cat bin/mailrestart.sh
1
2
3
4
5
6
#!/bin/sh

/usr/sbin/postfix reload
/usr/pkg/bin/doveadm reload

su imil -c 'echo "done"|mail -s "postfix and dovecot reloaded" imil'

direvent is started from /etc/rc.local as there’s no init script given with the package.

1
/usr/pkg/bin/direvent /usr/pkg/etc/direvent.conf

February 29, 2020

DragonFly BSD Digest In Other BSDs for 2020/02/29

Happy leap day!


February 22, 2020

DragonFly BSD Digest In Other BSDs for 2020/02/22

No theme this week cause I think I hit everything.

 


February 17, 2020

OS News NetBSD 9.0 released

The NetBSD Project is pleased to announce NetBSD 9.0, the seventeenth major release of the NetBSD operating system. This release brings significant improvements in terms of hardware support, quality assurance, security, along with new features and hundreds of bug fixes.

Support for the ARM architecture seems to be a major pillar of this new release.


February 15, 2020

DragonFly BSD Digest In Other BSDs for 2020/02/15

UNIX history as an accidental theme this week.

Update: NetBSD 9.0 is released.


February 14, 2020

NetBSD.org NetBSD 9.0 released

February 13, 2020

Unix Stack Exchange Difference between `pkgin` and `pkg_*`, and which to use?

Coming from Linux land, I assumed that pkgin is some sort of higher-level frontend for pkg_add et al, something like what apt is to dpkg or yum/dnf to rpm. But pkg_add appears to handle installation from network, dependencies, as well as automatic updates (the things that on Linux would be the responsibility of the front-ends rather than the underlying package tool), so now the roles of pkgin and (what I gather are the more traditional) pkg_ tools seem a bit unclear to me. The only difference that I'm presently aware of is that pkgin does not handle installation from sources.


February 03, 2020

Kimmo Suominen Updated sudo

I updated security/sudo to 1.8.31. A fix for CVE-2019-18634 is included.

Originally only versions before 1.8.26 were thought to be vulnerable. Later analysis, however, showed that versions 1.8.26 – 1.8.30 are also vulnerable.

Here’s what’s new:


January 27, 2020

Roy Marples openresolv-3.10.0 released

with the following changes:

ftp://roy.marples.name/pub/openresolv/openresolv-3.10.0.tar.xz
https://roy.marples.name/downloads/openresolv/openresolv-3.10.0.tar.xz

Roy Marples dhcpcd-8.1.6 released

with the following changes:

ftp://roy.marples.name/pub/dhcpcd/dhcpcd-8.1.6.tar.xz
http://roy.marples.name/downloads/dhcpcd/dhcpcd-8.1.6.tar.xz

This should be the last dhcpcd-8 release before dhcpcd-9 :)


January 25, 2020

Stack Overflow Alternatives to Vagrant for OS emulation/development [closed]

I am the maintainer psutil project. Since psutil supports different operating systems, I've been using Vagrant in order to to emulate them and develop on them from my Linux box. This worked flawlessly for many years (due to Vagrantfiles, which are great), but unfortunately I got to a point that Vagrant is no longer usable anymore. Problems derive mostly from the inability to mount shared folders, freezes when connecting via SSH and occasional incompatibilities between Vagrant and VirtualBox after a system upgrade. Are there viable alternatives to Vagrant? My working environment is Ubuntu and the systems I need to emulate are BSD*, MacOS and OpenSolaris (for Windows I use VirtualBox directly).


January 24, 2020

Kimmo Suominen Added patches to libxml2

I added patches to textproc/libxml2 from an upstream commit and an upstream pull request to address CVE-2020-7595 and CVE-2019-20388 respectively. Version 2.9.10nb1 includes the patches.


January 19, 2020

Kimmo Suominen Patch for HTTP Request Smuggling

I added a patch to www/nginx from an upstream commit to address CVE-2019-20372. Version 1.16.1nb2 includes the patch.


January 15, 2020

Roy Marples Anonymity Profiles for DHCP Clients aka RFC 7844

DHCP clients by default send a fair chunk of data which can identify you to the local DHCP server. In return they provide you with a stable IP address and configuration parameters.

At a bare minimum, the hardware address of the interface is sent - this is required to work.

So, how to solve this dilema of wanting total anonymity? The answer is to randomise the hardware address. This will happen when the carrier is down OR dhcpcd starts with the interface down. Then, dhcpcd will use this random hardware address to set a DUID LL which will be used inplace of any saved DUID and set the IAID to all zeros. This combo is used by DHCP and DHCPv6 to identify a lease. As this is randomised each time the carrier comes up you get a different IP address!

Try not to use this on an unstable link as it could drain the DHCP server of addresses :(

But we can't stop there! dhcpcd also sends some identifying options as well! For example, this is sent in the vendor class identifier:
dhcpcd-8.99.0:NetBSD-9.99.17:amd64:x86_64

It does not identify you or the device in anyway, but it does say what software is being used on which hardware. This could be used by DHCP servers to hand out a specific image to download and boot from TFTP for network boot clients.

Now, there are a gazzillion and one DHCP options out there - we don't know what you've configured. So dhcpcd will mask all of them when anonymous mode is activated, unless they are essential for enabling dhcpcd to work correctly on the network. But wait! What if you really want to leak something? Like say your on a corporate network that uses DHCP security and still want to remain anonymous? Well you can! Any request or option after the anonymous option in dhcpcd.conf is turned on. So the placing of the anonymous directive is important, unlike other dhcpcd options. So far this is the only implementation of RFC 7844 which does this :)

This is NOT enabled by default because most people want stable addresses AND a flappy link could drain addresses as disussed earlier.


January 11, 2020

Kimmo Suominen Cherry-picked patches for ncurses

In order to reduce the number of vulnerabilities on my systems, I added some patches to devel/ncurses (and devel/ncursesw) to address CVE-2018-19211, CVE-2019-17594, and CVE-2019-17595. Version 6.1nb7 includes the patches.


January 09, 2020

Roy Marples structure padding in C

Whilst developing Privilege Separation in dhcpcd, I had to come up with an IPC design for it. Of course, that involves creating structures.

So far, my structures in dhcpcd are long lived - or rather the scope is design to live outside of where it was created. As such they are created on the heap and are at the mercy of malloc. Generally I use calloc so that the whole area is inited to zero as uninitialised memory is bad.

So I decided to start out and see if I can just create the structures I need on the stack. Turns out I could! Yay! Now, how to you initialise a structure on the stack to all zeros? First let us consider this structure:

struct ps_addr {
    sa_family_t psa_family;
    union {
        struct in_addr psau_in_addr;
        struct in6_addr psau_in6_addr;
    } psa_u;
#define psa_in_addr     psa_u.psau_in_addr
#define psa_in6_addr    psa_u.psau_in6_addr
};

The first way is memset:

struct ps_addr psa;

memset(&psa, 0, sizeof(psa));
psa.psa_family = AF_INET;

But what if you could avoid memset? Luckily the C standard allows setting any member and will zero all other members. So we can do this:

struct ps_addr psa = { .psa_family = AF_INET };

Wow!!! So simple. This reduces binary size a fair bit. But then I turned on the Memory Sanitiser and boom, it crashed hard. Why?

The answer is simple - padding. Eric S Raymond gives a very good writeup about the problem. Basically, the standard will initialise any unintialised members to zero - but padding added for alignent isn't a member! So we need to ensure that our structure requires zero padding.

Here is the new struct:

struct ps_addr {
    sa_family_t psa_family;
    uint8_t psa_pad[4 - sizeof(sa_family_t)];
    union {
        struct in_addr psau_in_addr;
        struct in6_addr psau_in6_addr;
    } psa_u;
#define psa_in_addr     psa_u.psau_in_addr
#define psa_in6_addr    psa_u.psau_in6_addr
};

And it allows the former structure initialisation to work and memory sanitisers are happy - so happy days :) Now, if anyone can tell me what I can use instead of the magic number 4 in the above I'd be even happier!


January 08, 2020

Amitai Schlair NYCBUG: What is notqmail?

On Wednesday, January 8, I attended the New York City BSD User Group to present What is notqmail?, a perhaps not entirely surprising followup to my March talk. At the time, I’d been trying to avoid creating yet another qmail fork. This talk is about my failure — notqmail is alive and well — and about our success thus far.

Abstract:

We all use email, so we all use email servers. notqmail is software for running an email server. Someday, if we do a good job, some of the many articles about how and why to run your own will recommend notqmail.

notqmail is a community-driven fork of qmail, beginning where netqmail left off: providing stable, compatible, small releases to which existing qmail users can safely update. notqmail also aims higher: developing an extensible, easily packaged, and increasingly useful modern mail server.


Work with me

Would you personally benefit from an individualized session with an experienced, inquisitive, and empathetic conversation partner? Maybe you’re facing a challenging situation at work, a learning opportunity in some code — or both. Last week a new client went from “frustrated” to “energized” in the span of an hour. Get in touch: latentagility.com

Would your org benefit from a rare combination of technical coaching and impactful conversations? (Take other people’s word for it, not mine.) It’s not too late to book some time with me in 2020. Let’s talk about what fits for you.


January 07, 2020

Kimmo Suominen Restored correct psify source

I have restored fetching of correct upstream files for print/psify. Looks like we pointed to the wrong files for 13 years.


January 06, 2020

Unix Stack Exchange Is there anything similar to CrystalDiskMark for UNIX?

If you're shopping for an SSD, you've surely seen one of those screenshots from CrystalDiskMark with a few green squares, and the 2x4 matrix with results of doing read/write tests for the given hardware:

However, I've never seen anything similar for UNIX — at most what you get is a dd for sequential read/write tests, which gives no indication on the IOPS parameters of the hardware in question.

Is there anything similar to CrystalDiskMark for UNIX to perform various read/write tests like 4KiB Q8T8 etc?

I searched and found the following items in OpenBSD ports, but they seem rather stale (to say the least — randread is still hosted on SourceForge in 2020 and BYTE magazine has reportedly ceased online publication in 2013), and none of these tools make any mention of evaluating modern SSD performance, for which you probably have to have some sort of extra code to deal with the IO queues and threads or whatnot:


January 03, 2020

Roy Marples dhcpcd-8.1.5 released

with the following changes:

If you are suffering from IPv6 addresses not transitioning from the tentative state (regression from dhcpcd-8.1 on Linux) you will need to do one of the following after installing dhcpcd:

ftp://roy.marples.name/pub/dhcpcd/dhcpcd-8.1.5.tar.xz
http://roy.marples.name/downloads/dhcpcd/dhcpcd-8.1.5.tar.xz


December 21, 2019

Stack Overflow Rump unikernel compile successfully on a server but error on Ubuntu 19.04

I downloaded rumprun and followed the Tutorial to compile and install it.

git submodule update --init
CC=cc ./build-rr.sh hw

I have compiled successfully on a Ubuntu 18.04 system. When I copied the same code on that system to my system Ubuntu 19.04 to compile. It failed to compile. There were many error messages. Many of them are some warning. For example,

rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../sys/conf.h:129:19: error: cast between incompatible function types from 'int (*)(void)' to 'int (*)(dev_t,  int,  int,  struct lwp *)' {aka 'int (*)(long unsigned int,  int,  int,  struct lwp *)'} [-Werror=cast-function-type]
 #define noclose  ((dev_type_close((*)))enodev)
rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../net/bpf.c:184:13: note: in expansion of macro 'noclose'
  .d_close = noclose,
             ^~~~~~~
rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../sys/conf.h:130:18: error: cast between incompatible function types from 'int (*)(void)' to 'int (*)(dev_t,  struct uio *, int)' {aka 'int (*)(long unsigned int,  struct uio *, int)'} [-Werror=cast-function-type]
 #define noread  ((dev_type_read((*)))enodev)
rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../net/bpf.c:185:12: note: in expansion of macro 'noread'
  .d_read = noread,
            ^~~~~~

I searched for these compiling errors but I haven't found the same bugs. I don't know why this happens. How can I do to solve this problem? Thanks for your help!


December 13, 2019

Super User NetBSD - no pkg

After full installation of latest NetBSD I'm tried to launch pkgin, but received pkgin not found, also I've got same for pkgsrc. Then I've found, that there's no /usr/pkg location.

That's normal or I've did something wrong?


December 01, 2019

Server Fault ssh tunnel refusing connections with "channel 2: open failed"

All of a sudden (read: without changing any parameters) my netbsd virtualmachine started acting oddly. The symptoms concern ssh tunneling.

From my laptop I launch:

$ ssh -L 7000:localhost:7000 [email protected] -N -v

Then, in another shell:

$ irssi -c localhost -p 7000

The ssh debug says:

debug1: Connection to port 7000 forwarding to localhost port 7000 requested.
debug1: channel 2: new [direct-tcpip]
channel 2: open failed: connect failed: Connection refused
debug1: channel 2: free: direct-tcpip: listening port 7000 for localhost port 7000, connect from 127.0.0.1 port 53954, nchannels 3

I tried also with localhost:80 to connect to the (remote) web server, with identical results.

The remote host runs NetBSD:

bash-4.2# uname -a
NetBSD host 5.1_STABLE NetBSD 5.1_STABLE (XEN3PAE_DOMU) #6: Fri Nov  4 16:56:31 MET 2011  [email protected]:/m/obj/m/src/sys/arch/i386/compile/XEN3PAE_DOMU i386

I am a bit lost. I tried running tcpdump on the remote host, and I spotted these 'bad chksum':

09:25:55.823849 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 67, bad cksum 0 (->3cb3)!) 127.0.0.1.54381 > 127.0.0.1.7000: P, cksum 0xfe37 (incorrect (-> 0xa801), 1622402406:1622402421(15) ack 1635127887 win 4096 <nop,nop,timestamp 5002727 5002603>

I tried restarting the ssh daemon to no avail. I haven't rebooted yet - perhaps somebody here can suggest other diagnostics. I think it might either be the virtual network card driver, or somebody rooted our ssh.

Ideas..?


November 25, 2019

Super User What device does NetBSD use for a USB modem?

I'm testing some software on NetBSD 8.1 x86_64. The software opens a USB modem and issues AT commands. The software tested OK on Debian, Fedora, OS X, and OpenBSD. The software is having trouble on NetBSD.

NetBSD's dmesg shows:

umodem0 at uhub1 port 1 configuration 2 interface 0
umodem0: U.S.Robotics (0xbaf) USB Modem (0x303), rev 2.00/2.00, addr 2, iclass 2/2
umodem0: data interface 1, has CM over data, has break
umodem0: status change notification available
ucom0 at umodem0

If I am parsing the NetBSD man pages properly (which may not be the case), I should be able to access the modem via /dev/ucom0. Also see UMODEM(4) man page.

The test user is part of the dialer group. The software was not able to open /dev/ucom0, /dev/umodem0, ucom0 or umodem0. All open's result in No such file or directory. Additionally, there are no /dev/ttyACMn or /dev/cuaUn devices.

How do I access the modem on NetBSD?


November 17, 2019

Stack Overflow Compile only kernel module on NetBSD

Is there a way to compile only a kernel module on NetBSD? I can't seem to figure out how to do so without recompiling the entire kernel. Thanks!


October 25, 2019

Stack Overflow How can I make a NetBSD VM halt itself in google compute engine

I've got a batch job that I want to run in google compute engine on a NetBSD instance. I expected that I could just shutdown -hp now at the end of the job and the instance would be powered off. But when I do that it still remains in the running state according to the google cloud console and CLI. How do I make a NetBSD virtual machine in google cloud shut itself off when it's no longer needed?

Note: Google cloud SDK is not available on NetBSD


August 23, 2019

Unix Stack Exchange NetBSD - Unable to install pkgin

I'm running NetBSD on the Raspberry Pi 1 Model B.

uname -a
NetBSD rpi 7.99.64 NetBSD 7.99.64 (RPI.201703032010Z) evbarm

I'm trying to install pkgin but I'm receiving an error about version mismatch ...

pkg_add -f pkgin
pkg_add: Warning: package `pkgin-0.9.4nb4' was built for a platform:
pkg_add: NetBSD/earmv6hf 7.99.42 (pkg) vs. NetBSD/earmv6hf 7.99.64 (this host)
pkg_add: Warning: package `pkg_install-20160410nb1' was built for a platform:
pkg_add: NetBSD/earmv6hf 7.99.58 (pkg) vs. NetBSD/earmv6hf 7.99.64 (this host)
pkg_add: Can't create pkgdb entry: /var/db/pkg/pkg_install-20160410nb1: Permission denied
pkg_add: Can't install dependency pkg_install>=20130901, continuing
pkg_add: Warning: package `libarchive-3.3.1' was built for a platform:
pkg_add: NetBSD/earmv6hf 7.99.59 (pkg) vs. NetBSD/earmv6hf 7.99.64 (this host)
pkg_add: Can't create pkgdb entry: /var/db/pkg/libarchive-3.3.1: Permission denied
pkg_add: Can't install dependency libarchive>=3.2.1nb2, continuing
pkg_add: Can't create pkgdb entry: /var/db/pkg/pkgin-0.9.4nb4: Permission denied
pkg_add: 1 package addition failed

How can I install the correct version?


August 20, 2019

Super User How to run a Windowed JAR file over SSH without installing JRE and without root access on NetBsd?

First, I can use Java, but for what I want to achieve (building a database where othe only application supporting the format is in Java), I need 100Gb of RAM during 20 hours.

I have access to a server with the required RAM, but not as root and no JRE is available. The same is true for the Xorg libraries.

Here’s the uname :

8.0_STABLE NetBSD 8.0_STABLE (GENERIC) #0: Sun Feb 24 10:50:49 UTC 2019  [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC amd64

The Linux layer is installed, but nothing else is installed : not even Glibc, so the only applications which can be run are the ones which are statically compiled.

So not only Java isn’t Installed, but some of the require shared libraries are missing…
However, I have full write access to my $HOME directory, and I can run my own executables from there.

Is a way to convert a Jar file into a NetBsd Executable or Linux statically linked executable ? I have also the source code of the Jar file if compiling is an acceptable solution.

I only found about ɢᴄᴊ, but I’m unsure if Java 7 is supported…

Amitai Schlair Announcing notqmail

Running my own email server has its challenges. Chief among them: my favorite mail-server software hasn’t been updated since I started using it in 1998.

The qmail logo
qmail logo

Okay, that’s not entirely true. While qmail hasn’t been updated by its original author, a group of respected users created netqmail, a series of tiny updates that were informed, conservative, and careful. By their design, it was safe for everyone running qmail to follow netqmail, so everyone did. But larger changes in the world of email — authentication, encryption, and ever-shifting anti-spam techniques — remained as puzzles for each qmail administrator to solve in their own way. And netqmail hasn’t been updated since 2007.

One fork per person

In the interim, devotees have continued maintaining their own individual qmail forks. Some have shared theirs publicly. I’ve preferred the design constraints of making minimal, purpose-specific, and conflict-avoidant add-ons and patches. Then again, these choices are motivated by the needs of my qmail packaging, which I suppose is itself a de facto fork.

I’ve found this solo work quite satisfying. I’ve learned more C, reduced build-time complexity, added run-time configurability, and published unusually polished and featureful qmail packages for over 20 platforms. Based on these experiences, I’ve given dozens of workshops and talks. In seeking to simplify system administration for myself and others, I’ve become a better programmer and consultant.

Still, wouldn’t it be more satisfying if we could somehow pool our efforts? If, long after the end of DJB’s brilliant one-man show, a handful of us could shift how we relate to this codebase — and to each other — in order to bring a collaborative open-source effort to life? If, with netqmail as inspiration, we could produce safe updates while also evolving qmail to meet more present-day needs?

One fork per community

My subtle artwork
notqmail logo == qmail logo overlaid by a red circle with a slash through it

Say hello to notqmail.

Our first release is informed, conservative, and careful — but bold. It reflects our brand-new team’s rapid convergence on where we’re going and how we’ll get there. In the span of a few weeks, we’ve:

I say “bold” because, for all the ways we intend to hew to qmail tradition, one of our explicit goals is a significant departure. Back in the day, qmail’s lack of license, redistribution restrictions, technical barriers, and social norms made it hard for OS integrators to create packages, and hard for package users to get help. netqmail 1.06 expressed a desire to change this. In notqmail 1.07, we’ve made packaging much easier. (I’ve already updated pkgsrc from netqmail to notqmail, and some of my colleagues have prepared notqmail RPM and .deb packages.) Further improvements for packagers are part of what’s slated for 1.08.

What’s next

Looking much further ahead, another of our explicit goals is “Meeting all common needs with OS-provided packages”. We have a long way to go. But we couldn’t be off to a better start.

By our design, we believe we’ve made it safe for everyone running qmail to follow notqmail. We hope you’ll vet our changes carefully, then update your installations to notqmail 1.07. We hope you’ll start observing us as we continue the work. We hope you’ll discuss freely on the qmail mailing list. We hope you’ll be a part of the qmail revival in ways that are comfortable for you. And we hope that, in the course of time, notqmail will prove to be the community-driven open-source successor to qmail.


July 13, 2019

Jeremy C. Reed 2019-July-13 pfSense Essentials Book Writing

This week I received my printed proof from the printer and enabled it to be printed. It is now for sale at Amazon and Barnes and Noble,

I set an alarm to work on it very early a few days a week and it took me a few years. (I am blessed to only commute a few times a year, so I make sure that I don't waste that gifted time.)

This book was written using Docbook using NetBSD and vi. The print-ready book was generated with Dblatex version 0.3.10 with a custom stylesheet, pdfTeX 3.14159265-2.6-1.40.19 (Web2C 2018), and the TeX document production system installed via Tex Live and Pkgsrc. Several scripts and templates were created to help have a consistent document.

The book work was managed using the Subversion version control software. I carefully outlined my steps in utilizing the useful interfaces and identified every web and console interface. The basic writing process included adding over 350 special comment tags in the docbook source files that identified topics to cover and for every pfSense web interface PHP script (highlighting if they were main webpages from the pfSense menu). As content was written, I updated these special comments with a current status. A periodic script checked the docbook files and the generated book and reported on writing progress and current needs.

During this writing, nearly every interface was tested. In addition, code and configurations were often temporarily customized to simulate various pfSense behaviors and system situations. Most of the pfSense interface and low-level source code was studied, which helped with identifying pfSense configurations and features that didn't display in standard setups and all of its options. The software was upgraded several times and installed and ran in multiple VMs and hardware environments with many wireless and network cards, including with IPv6. In addition, third-party documentation and even source code was researched to help explain pfSense configurations and behaviors.

As part of this effort, I documented 352 bugs (some minor and some significant) and code suggestions that I found from code reading or from actual use of the system. (I need to post that.)

The first subversion commit for this book was in July 2014. It has commits in 39 different months with 656 commits total. The book's docbook source had 3789 non-printed comments and 56,193 non-blank lines of text. The generated book has over 180,000 words. My subversion logs show I have commits on 41 different Saturdays. Just re-reading with cleanup took me approximately 160 hours.


July 08, 2019

Server Fault Webserver farm with NFS share (autofs failure)

I am trying to set up the farm of webservers, consisting of the internal, external and worker servers.

  1. The actual sites content is stored on internal NFS server deep in internal network. All sites contents management is centralized.

  2. BSD-based external servers have Lighttpd doing all the HTTP/HTTPS job, serving static content. Main NFS share is auto-mounted via special path, like /net/server/export/www/site/ (via amd).

  3. Every Lighttpd have fastcgi parameters pointing to several worker servers, which have php-fpm working (for example). Different sites may require different php versions or arrangement, so www01 and www02 may serve site "A" having php-fpm over PHP 5.6 and www05 and www06 will serve site "B" having php-fpm over PHP 7.2.

  4. Every worker get requests for certain sites (one or more) with path /net/server/export/www/site and execute PHP or any other code. They also have amd (for BSD) and autofs (for Linux) working.

  5. For some sites Lighttpd may not forward fastcgi, but do proxying instead, so workers can have Apache or other web-server (even Java-based) working.

External servers are always BSD, internal servers too, but workers can be different upon actual needs.

This all work good when workers are BSD. If we are using Linux on workers - it stops working when share is automatically unmounted. When one tries to access the site he will get error 404. When I connect to server via ssh I will see no mounted share on "df -h". If I do any "ls" on /net/server/export - it is self-mounted as intended and site starts to work. On BSD-systems df show amd shares always mounted despite of 60 seconds dismount period.

I believe there is a difference between amd and autofs approach, php-fpm calls on Linux become some kind of "invisible" to autofs and do not cause auto-mount, because any other access to /net/server/ work at any time and do cause auto-mount. Also, this happens not with php-fpm only, Apache serving static content on auto-mounted NFS share behave same way.

Sorry for long description, but I tried to describe it good. The main question here - is anyone know why calls to /net/server may not cause auto-mount in autofs and how to prevent this behavior.

For lot of reasons I do not consider using static mounting, so this is not an option here. As for Linux versions - mostly it was tested on OEL 7.recent.


July 03, 2019

Super User Using a Console-only NetBSD VM

I am experimenting with NetBSD and seeing if I can get the Fenrir screenreader to run on it. However, I hit a snag post install; the console that I was using for the installation was working perfectly fine, however it stopped working alltogether once I completed the install. For reference, here is the line I used for virt-install:

virt-install --connect qemu:///system -n netbsd-testing \
             --ram 4096 --vcpus=8 \
             --cpu=host \
             -c /home/sektor/Downloads/boot-com.iso  \
             --os-type=netbsd --os-variant=netbsd8.0 \
             --disk=pool=devel,size=100,format=qcow2 \
             -w network=default --nographics 

When it asked me for the type of terminal I was using (this being the NetBSD install program), I accepted the default which was VT200. As I recall, I told it to use the BIOS for booting, and not any of the comm serial ports. Has anyone had any further experience with using no graphics on a Libvirt virtualized machine, and have any points as to how to get a working console?

Thanks.


March 07, 2019

Amitai Schlair NYCBUG: Maintaining qmail in 2019

On Wednesday, March 6, I attended New York City BSD User Group and presented Maintaining qmail in 2019. This one pairs nicely with my recent DevOpsDays Ignite talk about why and how to Run Your @wn Email Server! That this particular “how” could be explained in 5 minutes is remarkable, if I may say so myself. In this NYCBUG talk — my first since 2014 — I show my work. It’s a real-world, open-source tale of methodically, incrementally reducing complexity in order to afford added functionality.

My abstract:

qmail 1.03 was notoriously bothersome to deploy. Twenty years later, for common use cases, I’ve finally made it pretty easy. If you want to try it out, I’ll help! (Don’t worry, it’s even easier to uninstall.) Or just listen as I share the sequence of stepwise improvements from then to now — including pkgsrc packaging, new code, and testing on lots of platforms — as well as the reasons I keep finding this project worthwhile.

Here’s the video:


January 25, 2019

Amitai Schlair DevOpsDays NYC: Run Your @wn Email Server!

In late January, I was at DevOpsDays NYC in midtown Manhattan to present Run Your @wn Email Server!

My abstract:

When we’re responsible for production, it can be hard to find room to learn. That’s why I run my own email server. It’s still “production” — if it stays down, that’s pretty bad — but I own all the decisions, take more risks, and have learned lots. And so can you! Come see why and how to get started.

With one command, install famously secure email software. A couple more and it’s running. A few more and it’s encrypted. Twiddle your DNS, watch the mail start coming in, and start feeling responsible for a production service in a way that web hosting can’t match.


January 07, 2019

Amitai Schlair 2018Q4 qmail updates in pkgsrc

Happy 2019! Another three months, another stable branch for pkgsrc, the practical cross-platform Unix package manager. I’ve shipped quite a few improvements for qmail users in our 2018Q4 release. In three sentences:

  1. qmail-run gains TLS, SPF, IPv6, SMTP recipient checks, and many other sensible defaults.
  2. Most qmail-related packages — including the new ones used by qmail-run — are available on most pkgsrc platforms.
  3. rc.d-boot starts rc.conf-enabled pkgsrc services at boot time on many platforms.

In one:

It’s probably easy for you to run qmail now.

On this basis, at my DevOpsDays NYC talk in a few weeks, I’ll be recommending that everyone try it.

Try it

Here’s a demo on CentOS 7, using binary packages:

The main command I ran:

$ sudo env PKG_RCD_SCRIPTS=yes pkgin -y install qmail-run rc.d-boot

Here’s another demo on Debian 9, building from source packages:

The commands I ran:

$ cd ...pkgsrc/mail/qmail-run && make PKG_RCD_SCRIPTS=yes install
$ cd ../../pkgtools/rc.d-boot && make PKG_RCD_SCRIPTS=yes install

These improvements were made possible by acceptutils, my redesigned TLS and SMTP AUTH implementation that obviates the need for several large and conflicting patches. Further improvements are expected.

Here’s the full changelog for qmail as packaged in pkgsrc-2018Q4.

Removed

Updated

Added


March 17, 2018

Hubert Feyrer The adventure of rebuilding g4u from source
I was asked by a long-time g4u user on help with rebuilding g4u from sources. After pointing at the instructions on the homepage, we figured out that a few lose odds and ends didin't match. After bouncing some advices back and forth, I ventured into the frabjous joy of starting a rebuild from scratch, and quick enough ran into some problems, too.

Usually I cross-compile g4u from Mac OS X, but for the fun of it I did it on NetBSD (7.0-stable branch, amd64 architecture in VMware Fusion) this time. After waiting forever on the CVS checkout, I found that empty directories were not removed - that's what you get if you have -P in your ~/.cvsrc file.

I already had the hint that the "g4u-build" script needed a change to have "G4U_BUILD_KERNEL=true".

From there, things went almost smooth: building indicated a few files that popped up "variable may be used uninitialized" errors, and which -- thanks to -Werror -- bombed out the build. Fixing was easy, and I have no idea why that built for me on the release. I have sent a patch with the required changes to the g4u-help mailing list. (After fixing that I apparently got unsubscribed from my own support mailing list - thank you very much, Sourceforge ;)).

After those little hassles, the build worked fine, and gave me the floppy disk and ISO images that I expected:

>       ls -l `pwd`/g4u*fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u1.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u2.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u3.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u4.fs
>       ls -l `pwd`/g4u.iso
>       -rw-r--r--  2 feyrer  staff  6567936 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u.iso
>       ls -l `pwd`/g4u-kernel.gz
>       -rw-r?r--  1 feyrer  staff  6035680 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u-kernel.gz 
Next steps are to confirm the above changes as working from my faithful tester, and then look into how to merge this into the build instructions .

January 04, 2018

Hubert Feyrer NetBSD 7.1.1 released
On December 22nd, NetBSD 7.1.1 was released as premature christmas present, see the release annoucement.

NetBSD 7.1.1 is the first update with security and critical fixes for the NetBSD 7.1 branch. Those include a number of fixes for security advisories, kernel and userland.

Hubert Feyrer New year, new security advisories!
So things have become a bit silent here, which is due to reallife - my apologies. Still, I'd like to wish everyone following this here a Happy New Year 2018! And with this, a few new security advisories have been published:
Hubert Feyrer 34C3 talk: Are all BSDs created equally?
I haven't seen this mentioned on the NetBSD mailing lists, and this may be of interest to some - there was a talk about security bugs in the various BSDs at the 34th Chaos Communication Congress:

In summary, many reasons for bugs are shown in many areas of the kernel (system calls, file systems, network stack, compat layer, ...), and what has happened after they were made known to the projects.

As a hint, NetBSD still has a number of Security Advisories to publish, it seems. Anyone wants to help out the security team? :-)


June 22, 2017

Server Fault How to log ssh client connection/command?

I would like to know how i could log SSH command lines a user is using on a server. For exemple, if the user Alex on my server is doing the following set of commands :

$ cd /tmp
$ touch myfile
$ ssh [email protected]
$ ssh [email protected]
$ vim anotherfile
$ ssh [email protected]

I would like to log the ssh commands used on the server in a file which looks like :

[2014-07-25 10:10:10] Alex : ssh [email protected]
[2014-07-25 10:18:20] Alex : ssh [email protected]
[2014-07-25 11:15:10] Alex : ssh [email protected]

I don't care what he did during his ssh session, i just want to know WHEN and TO WHERE he made a connection to another server.

The user is not using bash and i would like to avoid manipulating .bash_history anyway as the user can modify it.

Any clue on this ?

Thank you :)

edit : to be more specific :

a user connects to a server A and then connects from the server A to server B. I want to track down to which server he connects through ssh from server A.


June 08, 2017

Hubert Feyrer g4u 2.6 released
After a five-year period for beta-testing and updating, I have finally released g4u 2.6. With its origins in 1999, I'd like to say: Happy 18th Birthday, g4u!

About g4u: g4u ("ghosting for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.

The past: When I started g4u, I had the task to install a number of lab machines with a dual-boot of Windows NT and NetBSD. The hype was about Microsoft's "Zero Administration Kit" (ZAK) then, but that did barely work for the Windows part - file transfers were slow, depended on the clients' hardware a lot (requiring fiddling with MS DOS network driver disks), and on the ZAK server the files for installing happened do disappear for no good reason every now and then. Not working well, and leaving out NetBSD (and everything elase), I created g4u. This gave me the (relative) pain of getting things working once, but with the option to easily add network drivers as they appeared in NetBSD (and oh they did!), plus allowed me to install any operating system.

The present: We've used g4u successfully in our labs then, booting from CDROM. I also got many donations from public and private instituations plus comanies from many sectors, indicating that g4u does make a difference.

In the mean time, the world has changed, and CDROMs aren't used that much any more. Network boot and USB sticks are today's devices of choice, cloning of a full disk without knowing its structure has both advantages but also disadvantages, and g4u's user interface is still command-line based with not much space for automation. For storage, FTP servers are nice and fast, but alternatives like SSH/SFTP, NFS, iSCSI and SMB for remote storage plus local storage (back to fun with filesystems, anyone? avoiding this was why g4u was created in the first place!) should be considered these days. Further aspects include integrity (checksums), confidentiality (encryption). This leaves a number of open points to address either by future releases, or by other products.

The future: At this point, my time budget for g4u is very limited. I welcome people to contribute to g4u - g4u is Open Source for a reason. Feel free to get back to me for any changes that you want to contribute!

The changes: Major changes in g4u 2.6 include:

The software: Please see the g4u homepage's download section on how to get and use g4u.

Enjoy!


February 09, 2017

BSD Talk bsdtalk266 - The nodes take over
We became tired of waiting. File Info: 7Min, 3MB. Ogg Link: https://archive.org/download/bsdtalk266/bsdtalk266.ogg

July 08, 2016

Frederic Cambus NetBSD on the CubieBoard2

Here are some notes on installing and running NetBSD/evbarm on the AllWinner A20 powered CubieBoard2. I bought this board a few weeks ago for its SATA capabilities, despite the fact that there are now cheaper boards with more powerful CPUs.

Required steps for creating a bootable micro SD card are detailed on the NetBSD Wiki, and a NetBSD installation is required to run mkubootimage.

I used an USB to TTL serial cable to connect to the board and create user accounts. Do not be afraid of serial, as it has in fact only advantages: there is no need to connect an USB keyboard nor an HDMI display, and it also brings back nice memories.

Connecting using cu (from my OpenBSD machine):

cu -s 115200 -l /dev/cuaU0

Device name might be different when using cu on other operating systems.

Adding a regular user in the wheel group:

useradd -m -G wheel username

Adding a password to the newly created user and changing default shell to ksh:

passwd username
chpass -s /bin/ksh username

Installing and configuring pkgin:

export PKG_PATH=http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/earmv7hf/7.0/All
pkg_add pkgin
echo $PKG_PATH > /usr/pkg/etc/pkgin/repositories.conf
pkgin update

Finally, here is a dmesg for reference purposes:

Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 7.0.1 (CUBIEBOARD.201605221355Z)
total memory = 1024 MB
avail memory = 1008 MB
sysctl_createv: sysctl_create(machine_arch) returned 17
kern.module.path=/stand/evbarm/7.0/modules
timecounter: Timecounters tick every 10.000 msec
mainbus0 (root)
cpu0 at mainbus0 core 0: 912 MHz Cortex-A7 r0p4 (Cortex V7A core)
cpu0: DC enabled IC enabled WB disabled EABT branch prediction enabled
cpu0: isar: [0]=0x2101110 [1]=0x13112111 [2]=0x21232041 [3]=0x11112131, [4]=0x10011142, [5]=0
cpu0: mmfr: [0]=0x10101105 [1]=0x40000000 [2]=0x1240000 [3]=0x2102211
cpu0: pfr: [0]=0x1131 [1]=0x11011
cpu0: 32KB/32B 2-way L1 VIPT Instruction cache
cpu0: 32KB/64B 4-way write-back-locking-C L1 PIPT Data cache
cpu0: 256KB/64B 8-way write-through L2 PIPT Unified cache
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
vfp0: mvfr: [0]=0x10110222 [1]=0x11111111
cpu1 at mainbus0 core 1
armperiph0 at mainbus0
armgic0 at armperiph0: Generic Interrupt Controller, 160 sources (151 valid)
armgic0: 32 Priorities, 128 SPIs, 7 PPIs, 16 SGIs
armgtmr0 at armperiph0: ARMv7 Generic 64-bit Timer (24000 kHz)
armgtmr0: interrupting on irq 27
timecounter: Timecounter "armgtmr0" frequency 24000000 Hz quality 500
awinio0 at mainbus0: A20 (0x1651)
awingpio0 at awinio0
awindma0 at awinio0: DMA
awindma0: interrupting on irq 59
awincnt0 at awinio0
timecounter: Timecounter "CNT64" frequency 24000000 Hz quality 900
com0 at awinio0 port 0: ns16550a, working fifo
com0: console
awindebe0 at awinio0 port 0: Display Engine Backend (BE0)
awintcon0 at awinio0 port 0: LCD/TV timing controller (TCON0)
awinhdmi0 at awinio0: HDMI 1.3
awinwdt0 at awinio0: default period is 10 seconds
awinrtc0 at awinio0: RTC
awinusb0 at awinio0 port 0
awinusb0: no restrict gpio found
ohci0 at awinusb0: OHCI USB controller
ohci0: OHCI version 1.0
usb0 at ohci0: USB revision 1.0
ohci0: interrupting on irq 96
ehci0 at awinusb0: EHCI USB controller
ehci0: EHCI version 1.0
ehci0: companion controller, 1 port each: ohci0
usb1 at ehci0: USB revision 2.0
ehci0: interrupting on irq 71
awinusb1 at awinio0 port 1
awinusb1: no restrict gpio found
ohci1 at awinusb1: OHCI USB controller
ohci1: OHCI version 1.0
usb2 at ohci1: USB revision 1.0
ohci1: interrupting on irq 97
ehci1 at awinusb1: EHCI USB controller
ehci1: EHCI version 1.0
ehci1: companion controller, 1 port each: ohci1
usb3 at ehci1: USB revision 2.0
ehci1: interrupting on irq 72
motg0 at awinio0: OTG
motg0: interrupting at irq 70
motg0: no restrict gpio found
motg0: Dynamic FIFO sizing detected, assuming 16Kbytes of FIFO RAM
usb4 at motg0: USB revision 2.0
awinmmc0 at awinio0 port 0: SD3.0 (DMA)
awinmmc0: interrupting at irq 64
ahcisata0 at awinio0: AHCI SATA controller
ahcisata0: interrupting on irq 88
ahcisata0: ignoring broken port multiplier support
ahcisata0: AHCI revision 1.10, 1 port, 32 slots, CAP 0x6f24ff80<CCCS,PSC,SSC,PMD,SAM,ISS=0x2=Gen2,SCLO,SAL,SALP,SSS,SSNTF,SNCQ>
atabus0 at ahcisata0 channel 0
awiniic0 at awinio0 port 0: Marvell TWSI controller
awiniic0: interrupting on irq 39
iic0 at awiniic0: I2C bus
awge0 at awinio0: Gigabit Ethernet Controller
awge0: interrupting on irq 117
awge0: Ethernet address: 02:0a:09:03:27:08
rlphy0 at awge0 phy 1: RTL8201L 10/100 media interface, rev. 1
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
awinac0 at awinio0: CODEC
audio0 at awinac0: full duplex, playback, capture, mmap, independent
awinhdmiaudio0 at awinio0: HDMI 1.3
audio1 at awinhdmiaudio0: half duplex, playback, mmap
awinnand0 at awinio0
awinir0 at awinio0 port 0: IR
awinir0: interrupting on irq 37
cir0 at awinir0
gpio0 at awingpio0: 18 pins
gpio1 at awingpio0: 25 pins
gpio2 at awingpio0: 28 pins
gpio3 at awingpio0: 12 pins
gpio4 at awingpio0: 12 pins
gpio5 at awingpio0: 28 pins
gpio6 at awingpio0: 22 pins
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
cpu1: 912 MHz Cortex-A7 r0p4 (Cortex V7A core)
cpu1: DC enabled IC enabled WB disabled EABT branch prediction enabled
cpu1: isar: [0]=0x2101110 [1]=0x13112111 [2]=0x21232041 [3]=0x11112131, [4]=0x10011142, [5]=0
cpu1: mmfr: [0]=0x10101105 [1]=0x40000000 [2]=0x1240000 [3]=0x2102211
cpu1: pfr: [0]=0x1131 [1]=0x11011
cpu1: 32KB/32B 2-way L1 VIPT Instruction cache
cpu1: 32KB/64B 4-way write-back-locking-C L1 PIPT Data cache
cpu1: 256KB/64B 8-way write-through L2 PIPT Unified cache
vfp1 at cpu1: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
vfp1: mvfr: [0]=0x10110222 [1]=0x11111111
sdmmc0 at awinmmc0
uhub0 at usb0: Allwinner OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1 at usb1: Allwinner EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 1 port with 1 removable, self powered
uhub2 at usb2: Allwinner OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 1 port with 1 removable, self powered
uhub3 at usb3: Allwinner EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub3: 1 port with 1 removable, self powered
uhub4 at usb4: Mentor Graphics MOTG root hub, class 9/0, rev 1.00/1.00, addr 1
uhub4: 1 port with 1 removable, self powered
ld0 at sdmmc0: <0x02:0x544d:SA08G:0x14:0x12436e27:0x0e6>
ld0: 7388 MB, 3752 cyl, 64 head, 63 sec, 512 bytes/sect x 15130624 sectors
ld0: 4-bit width, bus clock 50.000 MHz
boot device: ld0
root on ld0a dumps on ld0b
root file system type: ffs

May 31, 2016

BSD Talk bsdtalk265 - Sunset on BSD
A brief description of playing around with SunOS 4.1.4, which was the last version of SunOS to be based on BSD. File Info: 17Min, 8Mb Ogg Link: https://archive.org/download/bsdtalk265/bsdtalk265.ogg

April 30, 2016

BSD Talk bsdtalk264 - Down the Gopher Hole
Playing around with the gopher protocol.   Description of gopher from the 1995 book "Student's Guide to the Internet" by David Clark. Also, at the end of the episode is audio from an interview with Mark McCahilll and Farhad Anklesaria that can be found at https://www.youtube.com/watch?v=oR76UI7aTvs Check out http://gopher.floodgap.com/gopher/ File Info: 27 Min, 13 MB. Ogg Link:https://archive.org/download/bsdtalk264/bsdtalk264.ogg