NetBSD Planet


May 20, 2020

Amitai Schlair notqmail 1.08 released

notqmail logo

I’m pleased to announce notqmail 1.08, the latest update to notqmail. It addresses security vulnerabilities in qmail 1.03 reported yesterday by Qualys. As a fork, we’ve inherited qmail’s relatively few bugs; as an active qmail fork, we’ve addressed the vulnerabilities with a timely release; and as the community-driven open-source successor to qmail, we show our work.

The 1.08 release also includes many other small improvements we’ve made since 1.07, including fewer compiler warnings, less dead code, published intent to remove more presumed-dead code, my QMAILREMOTE patch, and our first few unit tests. For more detail, see the notqmail 1.08 release notes.


May 16, 2020

DragonFly BSD Digest In Other BSDs for 2020/05/16

We need a “BSD Games” site.


May 13, 2020

Server Fault Non-standard IP address with dashes

I ran the who command on a shared NetBSD box, and this weird user IP came up:

<redacted> pts/33   May 13 02:13  (XXX.XXX.XXX.XXX)
<redacted> pts/35   May 12 20:59  (202-172-110-147-)
<redacted> pts/36   May  6 20:36  (XXX.XXX.XXX.XXX)

I've never seen an IP like that. Obviously, ping 202-172-110-147- will complain with "Cannot resolve ... (Unknown host)".

There was a similar question posted 7 years ago, which posited that it was a non-standard way of denoting IP ranges, but seeing there's a - at the end of the address, it doesn't seem like a similar thing.


Edit:

I've tried reverse DNS with nslookup 202-172-110-147-, which errors with "** server can't find 202-172-110-147-: NXDOMAIN"

Doing w <user> returns:

9:49AM  up 89 days,  7:46, 1 user, load averages: 0.23, 0.18, 0.17
USER          TTY     FROM              [email protected]  IDLE WHAT
<redacted>    pts/35  202-172-110-147- Tue08PM  4:13

Edit 2: This is on NetBSD, not Linux like I mentioned at the beginning (I thought the box was Linux):

$ uname -rsv
NetBSD 8.1 NetBSD 8.1 (GENERIC) #0: Fri May 31 08:43:59 UTC 2019  [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC

Edit 3: Following @NStorm update, I ran w -n to display network addresses as numbers. I still see the same result

$ w -n <user>
 9:57AM  up 89 days,  7:54, 1 user, load averages: 0.12, 0.12, 0.14
USER          TTY     FROM              [email protected]  IDLE WHAT
<redacted>    pts/35  202-172-110-147- Tue08PM  4:22

May 09, 2020

DragonFly BSD Digest In Other BSDs for 2020/05/09

Good mix of history and current BSDs this week.


May 07, 2020

DragonFly BSD Digest BSD Now 349: Entropy Overhaul

This week’s BSD Now is all FreeBSD and NetBSD news items – go read/listen; I have no special items to pick out but it’s all enjoyable.

NetBSD Blog Announcing Google Summer of Code 2020 projects

Google Summer of Code logo We are very happy to announce The NetBSD Foundation Google Summer of Code 2020 projects:

The community bonding period - where students get in touch with mentors and community - started on May 4 and will go on until June 1. The coding period will be June 1 to August 24.

Please welcome all our students and a big good luck to students and mentors!

A big thank you to Google and The NetBSD Foundation organization mentors and administrators!

Looking forward to having another nice Google Summer of Code!


May 05, 2020

Unix Stack Exchange NetBSD/FreeBSD and TP-Link TL-WN727N: Second encounter

Moved from SU.
First part at https://superuser.com/questions/975564/netbsd-and-tp-link-tl-wn727n-atheros-ar9271-or-ralink-rt5370

So, I've installed NetBSD 7 and device shown again as ugen(ugein, lol).

ugen0 at uhub4 port 8 
ugen0: Mediatek 802.11 n WLAN, rev 2.01/00, addr 2

Then I'm installed FreeBSD 10.2 and ugen again.

usbconfig gives me ugen4.3: <product 0x7601 vendor 0x148f> at usbus4, cfg=0 md=HOST spd=HIGH(480Mbps) pwr=ON (90ma)

So, what's next? Buying new dongle is a last thing, which I'll make.


May 04, 2020

NetBSD Blog The GNU GDB Debugger and NetBSD (Part 2)
The NetBSD team of developers maintains two copies of GDB:

The base-system version of GDB (GPLv3) still relies on a set of local patches. I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.

Over the past month I've reimplemented debugging support for multi-threaded programs and upstreamed the support. It's interesting to note that the old support relied on GDB tracking only a single inferior process. This caused the need to reimplement the support and be agnostic to the number of traced processes. Meanwhile the upstream developers introduced new features for multi-target tracing and a lot of preexisting code broke and needed resurrection. This affected also the code kept in the GDB basesystem version. Additionally over the past 30 days, I've also developed new CPU-independent GDB features that were for a long time on a TODO list for NetBSD.

After the past month NetBSD has now a decent and functional GDB support in the mainline. It's still not as featured as it could and CPU-specific handling will need a dedicated treatment.

Signal conversions

GDB maintains an internal representation of signals and translates e.g. SIGTRAP to GDB_SIGNAL_TRAP. The kernel independent management of signal names is used by the GDB core and requires translation on the border of kernel-specific implementation. So far, the NetBSD support relied on an accidental overlap of signal names between the GDB core and the NetBSD definitions, while this worked for some signals it wasn't matching for others. I've added a patch with appropriate NetBSD->GDB and GDB->NetBSD signal number conversions and enabled it in all NetBSD CPU-specific files.

Later, newly added code respects now these signal conversions in the management of processes.

Threading support

I've implemented the NetBSD specific methods for dealing with threads. Previously the pristine version of GDB was unaware about threading on NetBSD and the basesystem GDB relied on local patches that needed reimplementation to meet the expectations (especially rewrite to C++) of the upstream maintainers.

I have upstreamed this with the following commit:

Implement basic threading support in the NetBSD target

Use sysctl(3) as the portable interface to prompt NetBSD threads on
all supported NetBSD versions. In future newer versions could switch
to PT_LWPSTATUS ptrace(2) API that will be supported on NetBSD 10.0
and newer.

Implement as part of nbsd_nat_target:
 - thread_name()         - read descriptive thread name
 - thread_alive()        - check whether a thread is alive
 - post_attach()         - updates the list of threads after attach
 - update_thread_list()  - updates the list of threads
 - pid_to_str()          - translates ptid to a descriptive string

There are two local static functions:
 - nbsd_thread_lister()  - generic LWP lister for a specified pid
 - nbsd_add_threads()    - utility to update the list of threads

Now, GDB on NetBSD can attach to a multithreaded process, spawn
a multithreaded process, list threads, print their LWP+PID numbers
and descriptive thread names.

ELF symbol resolver

The NetBSD operating system relies on the ELF file format for native applications.

One of the features in GDB is to skip single-stepping over the internal ELF loader code. When a user of GDB instructs ther debugger to step a line in a source code, and whenever we land into an unresolved symbol in the GOT table, we could step internal code of the ELF loader, for the first usage of a public symbol, which is not necessarily wrong, but could be confusing. This is typically worked around with a generic code that tries to detect such scenario examining among others the code sections of the stepped code, but the default fallback is not functional on every NetBSD port, especially Alpha and ARM. The new code merged into GDB uses the same logic for all NetBSD ports, trying to detect the _rtld_bind_start symbol and act acordingly in case of detecting it.

SVR4 psABI parsers of AUXV entries

The ELF format ships with a mechanism to transfer certain kernel level information to the user process. AUXV is a key-value array located on the stack and available to an ELF loader. While Linux uses a different format than the one specified in SVR4 psABI, NetBSD follows the standard and stores the key value in a 32-bit integer always. This caused breakage on 64-bit CPUs and the NetBSD developers used to patch the Linux AUXV code to be compatible with the NetBSD behavior. I've added a dedicated function for the NetBSD AUXV handling and switched to it all NetBSD CPUs.

Process information (info proc)

As documented by the GDB project: Some operating systems provide interfaces to fetch additional information about running processes beyond memory and per-thread register state. If GDB is configured for an operating system with a supported interface, the command info proc is available to report information about the process running your program, or about any process running on your system.

Previously the info proc functionality was implemented only for Linux and FreeBSD. I've implemented support for the following commands:

All of these pieces of information are retrieved with the sysctl(3) interface. Example of an execution of the command is below:

(gdb) info proc all
process 26015
cmdline = '/usr/bin/cal'
cwd = '/public/binutils-gdb-netbsd'
exe = '/usr/bin/cal'
Mapped address spaces:

          Start Addr           End Addr       Size     Offset   Flags   File
            0x200000           0x204000     0x4000        0x0  r-x C-PD /usr/bin/cal
            0x404000           0x405000     0x1000     0x4000  r-- C-PD /usr/bin/cal
            0x405000           0x406000     0x1000        0x0  rw- C-PD 
      0x7f7ff6c00000     0x7f7ff6c10000    0x10000        0x0  rw- C-PD 
      0x7f7ff6c10000     0x7f7ff6db0000   0x1a0000        0x0  rw- CNPD 
      0x7f7ff6db0000     0x7f7ff6dc0000    0x10000        0x0  rw- C-PD 
      0x7f7ff6dc0000     0x7f7ff7000000   0x240000        0x0  rw- CNPD 
      0x7f7ff7000000     0x7f7ff7010000    0x10000        0x0  rw- C-PD 
      0x7f7ff7010000     0x7f7ff7200000   0x1f0000        0x0  rw- CNPD 
      0x7f7ff7200000     0x7f7ff7260000    0x60000        0x0  r-x CNPD /lib/libc.so.12.215
      0x7f7ff7260000     0x7f7ff7270000    0x10000    0x60000  r-x C-PD /lib/libc.so.12.215
      0x7f7ff7270000     0x7f7ff73c6000   0x156000    0x70000  r-x CNPD /lib/libc.so.12.215
      0x7f7ff73c6000     0x7f7ff75c6000   0x200000   0x1c6000  --- CNPD /lib/libc.so.12.215
      0x7f7ff75c6000     0x7f7ff75d1000     0xb000   0x1c6000  r-- C-PD /lib/libc.so.12.215
      0x7f7ff75d1000     0x7f7ff75d7000     0x6000   0x1d1000  rw- C-PD /lib/libc.so.12.215
      0x7f7ff75d7000     0x7f7ff75f0000    0x19000        0x0  rw- C-PD 
      0x7f7ff75f0000     0x7f7ff76e0000    0xf0000        0x0  rw- CNPD 
      0x7f7ff76e0000     0x7f7ff76f0000    0x10000        0x0  rw- C-PD 
      0x7f7ff76f0000     0x7f7ff77e0000    0xf0000        0x0  rw- CNPD 
      0x7f7ff77e0000     0x7f7ff77f8000    0x18000        0x0  rw- C-PD 
      0x7f7ff7800000     0x7f7ff780e000     0xe000        0x0  r-x CNPD /lib/libterminfo.so.2.0
      0x7f7ff780e000     0x7f7ff7a0d000   0x1ff000     0xe000  --- CNPD /lib/libterminfo.so.2.0
      0x7f7ff7a0d000     0x7f7ff7a0e000     0x1000     0xd000  r-- C-PD /lib/libterminfo.so.2.0
      0x7f7ff7a0e000     0x7f7ff7a0f000     0x1000     0xe000  rw- C-PD /lib/libterminfo.so.2.0
      0x7f7ff7c00000     0x7f7ff7c0f000     0xf000        0x0  r-x C-PD /libexec/ld.elf_so
      0x7f7ff7c0f000     0x7f7ff7e0f000   0x200000        0x0  --- CNPD 
      0x7f7ff7e0f000     0x7f7ff7e10000     0x1000     0xf000  rw- C-PD /libexec/ld.elf_so
      0x7f7ff7e10000     0x7f7ff7e11000     0x1000        0x0  rw- C-PD 
      0x7f7ff7eed000     0x7f7ff7eff000    0x12000        0x0  rw- C-PD 
      0x7f7ff7eff000     0x7f7fffbff000  0x7d00000        0x0  --- CNPD 
      0x7f7fffbff000     0x7f7fffff0000   0x3f1000        0x0  rw- CNPD 
      0x7f7fffff0000     0x7f7ffffff000     0xf000        0x0  rw- C-PD 
Name: cal
State: STOP
Parent process: 11837
Process group: 26015
Session id: 15656
TTY: 1288
TTY owner process group: 11837
User IDs (real, effective, saved): 1000 1000 1000
Group IDs (real, effective, saved): 100 100 100
Groups: 100 0 5
Minor faults (no memory page): 292
Major faults (memory page faults): 0
utime: 0.000000
stime: 0.003510
utime+stime, children: 0.000000
'nice' value: 20
Start time: 1588600926.724211
Data size: 8 kB
Stack size: 8 kB
Text size: 16 kB
Resident set size: 1264 kB
Maximum RSS: 2000 kB
Pending Signals: 00000000 00000000 00000000 00000000
Ignored Signals: 98488000 00000000 00000000 00000000
Caught Signals: 00000000 00000000 00000000 00000000

Event handling

I've implemented event handling for the following trap types:

While there, I have added proper support for ::wait () and ::resume () methods.

Syscall entry/exit tracing

I've added a support for syscall entry/exit breakpoint traps. There used to be some support for this mode in the basesystem GDB, however we missed a list of mapping of syscall number to syscall name. I've borrowed the script from FreeBSD to generate netbsd.xml with the mapping, based on definitions in /usr/include/sys/syscall.h. This approach of mapping syscalls for NetBSD is imperfect as the internal names are not stable and change whenever a syscall is versioned. There is no ideal approach to handle this (e.g. debugging programs on NetBSD 9.0 and NetBSD 10.0 can behave differently), and end-users will need to deal with it.

Threading events

As threading support is mandatory these days, I have implemented and upstreamed support for thread creation and thread exit events.

Other changes

As in general, I oppose to treating all BSD Operating Systems under the same ifdef in existing software, as it leads to conditionals like #if defined(AllBSDs) && !defined(ThisBSD) and/or a spaghetti of #define symbol renames. There was an attempt to define support for all BSDs in a shared file inf-ptrace.c, however in the end developers pushing for that reimplemented support part of the code for their kernels in private per-OS files. This left a lot of shared, sometimes unneeded code in the common layer.

I was kind to fix the build of OpenBSD support in GDB (and only build-tested) and moved OpenBSD-specific code from inf-ptrace.c to obsd-nat.c. Code that was no longer needed for OpenBSD in inf-ptrace.c (as it was reimplemented in obsd-nat.c) and in theory shared with NetBSD was removed.

My intention is to restrict common shared code of GDB to really common parts and whenever kernels differ, implement their specific handling in dedicated files. There are still some hacks left in the GDB shared code (even in inf-ptrace.c) for long removed kernels that obfuscate the support... especially the Gould NP1 support from the 1980s and definitely removed in 2000.

Plan for the next milestone

Finish and upstream operational support of follow-fork, follow-vfork and follow-spawn events. Rewrite the gdbserver support and submit upstream.


May 02, 2020

DragonFly BSD Digest In Other BSDs for 2020/05/02

Even though this is BSD-themed, there’s some offbeat items in here.


May 01, 2020

NetBSD.org New Developer in April 2020

April 27, 2020

NetBSD Blog Improving libossaudio, and the future of OSS in NetBSD

There's two ways user applications can communicate with the kernel audio layer in NetBSD:

Linux drifted away from OSS and towards ALSA due to licensing disagreements.

Because of this drift, we're seeing increasing problems with OSS adoption today, even if the licensing concerns are no longer relevant, and other implementations of OSS have surpassed the original Linux OSSv3 implementation as far as their feature set and usability are concerned.

So, in NetBSD, it's recommended to use the native API for new code and only rely on the OSS layer for compatibility with existing code.

I spent a while working on third-party software to improve support for native NetBSD audio. These included Firefox, SDL, PortAudio, ffmpeg (working with [email protected]), and more.

However, I've turned my attention to the OSS translation layer. Since a lot of older and less popular software still relies on it, I wanted to go over the OSSv4 specification and iron out surprising differences.

Audacity/PortAudio's OSS usage is strange

I should note that most of these fixes were to enable Audacity to work without patching. Audacity is interesting because it hits a lot of edge cases as far as OSS API usage is concerned. Once I fixed the most notable issues, I made sure Audacity also supported the native API. Writing the necessary PortAudio glue for Sun/NetBSD audio and implementing these fixes took approximately two days.

Incompatibility 1 – SNDCTL_DSP_SPEED

[Out of range sample rates are now handled properly by the OSS layer in NetBSD-current.]

The NetBSD 9 kernel supports sample rates up to 192kHz. Specify anything higher, and NetBSD's audio API returns an error code and keeps the sample rate at its original value, or the legacy default of 8000 Hz (not particularly useful with modern devices).

However, OSS applications expected setting the sample rate to always succeed. The specification states that the actual set sample value may be an approximation and will not always use the exact requested value. So, if the requested value is out of range, NetBSD will now return as if the call succeeded, and set the sample rate to the current configured hardware rate (usually some multiple of 48kHz).

During its startup process, Audacity requested an overly high sample rate of 384kHz. This is well above the maximum supported. I'm still not sure why it does this, because it later configures the audio device to standard CD rate, but it meant that Audacity couldn't properly start without our local patches.

Incompatibility 2 – SNDCTL_DSP_CHANNELS

[Out of range channel numbers are now handled properly by the OSS layer in NetBSD-current.]

This was a very simple fix, similar to that of SNDCTL_DSP_SPEED. The NetBSD kernel supports between 1 and 12 channels for audio playback. Most commonly 1 is mono, 2 is stereo, and higher numbers are used with surround sound systems. The limit of 12 comes from the USB audio device specification.

If an out of range number is specified, libossaudio will now set the channel count to the currently configured number in use at the hardware level.

However, we encounter a more difficult difference between OSS and NetBSD audio when using the audio device in full duplex (recording from and playing back to the same device simultaneously). If your mic is mono and your speakers aren't, how do you set the channel counts to different numbers in OSS? You can't. There is one ioctl for setting both the recording and playback channels. In the native API, this is possible by setting info.record.channels and info.play.channels separately. We should ensure that the recording channels are always duplicated to be same as the number of playback channels.

Incompatibility 3 – SNDCTL_DSP_SETTRIGGER

[NetBSD-current now implements SNDCTL_DSP_SETTRIGGER.]

SNDCTL_DSP_SETTRIGGER is a somewhat more obscure part of the OSS API, in that it's only really useful if you are using poll() or another event notification mechanism on an audio device before performing any I/O, or you're performing I/O via mmap(), neither being particularly common in practice. It has the ability to force initialisation of playback/recording for the device if this isn't already the case.

In terms of the native API, this means that playback/recording becomes unpaused.

Previously in NetBSD, this part of the OSS API wasn't implemented and simply did nothing. However, it became obviously needed due to an incompatible change in NetBSD 9, as discussed on tech-kern.

Basically, we needed recording to be properly triggered without a read() so a few applications using poll() without prior I/O wouldn't block indefinitely.

Incompatibility 4 – SNDCTL_DSP_SETPLAYVOL

OSSv4 has special bits to manipulate the volume of an individual stream in an application while doing all the maths for this inside the kernel.

We don't support this properly yet (but reasonably could)... so code needs to be modified to do the volume manipulation in the application, or the OSSv4 support disabled.

I've only found a couple of applications that try to use this feature (audacious, qmmp). Currently, they're configured to avoid using OSSv4 and layer the audio through SDL or qt-multimedia instead.

I've at least fixed SNDCTL_DSP_GETPLAYVOL to return conforming values. NetBSD audio uses 0-255 for the gain of all channels. OSSv4 uses a range of 0-100 and encodes two channels into an integer, which is very odd in my opinion, and also limits surround sound support.

The future of libossaudio in NetBSD?

Hopefully, after my changes, OSS compatibility is in a much better shape when dealing with unusual parameters and uncommon API usage. The quality of the code also improves – in the process of this work, [email protected] pointed me towards a related information leak in the Linux OSSv3 compatibility layer in the kernel, and I was able to deal with it properly after looking at the OSS specification and Linux headers. All the fixes should be pulled up to 9-stable.

However, I'd personally like to eventually reach a point where we no longer need libossaudio. I've been writing a lot of code towards this goal.

In many cases, the applications relying on it could be easily modified or told to use libao/SDL2/PortAudio/OpenAL/etc instead, which all have native NetBSD audio support.

OSS aside...

We probably need to start thinking about supporting 24-bit PCM in the kernel, since I've found a few audio players that can't handle making the samples 32-bit before writing them to the device. The Sun audio implementation in Solaris has supported this for a long time now.


April 24, 2020

DragonFly BSD Digest DragonFly, QEMU, NVMM, NetBSD/adm64 9

Charlotte Koch sent me this link some time ago and I’ve been remiss in not posting it: DragonFly through QEMU, using NVMM, on NetBSD.


April 21, 2020

Kimmo Suominen Serial console for NetBSD

I’ve been switching my NetBSD machines to using a serial console lately, as it is easier to copy output from it (no more screen shots). I would still need to figure out how to setup rtty1 to log console output, so unexpected output could also be inspected later (hello, panic).

On the NetBSD VM its bootblocks must be reinstalled to select a serial port as the console. I’ve been using the auto option, so I can easily switch between the VGA and serial consoles. I set speed to zero, which will use the speed configured in BIOS. The only available “speed” with qemu is 115200 bps.

installboot -v -o console=auto,speed=0 /dev/rsd0a /usr/mdec/bootxx_ffsv2

In Proxmox a serial port must be added to the VM on the Hardware tab. It is important to leave the Display setting at Default (or VGA) — for some reason input from the serial console will not work in the boot menu otherwise. (It will work in BIOS and at the login prompt.)


  1. Also known as remote-tty in Debian. 


April 16, 2020

Brett Lymn Thomas E. Dickey on NetBSD curses


I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses.  It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match.  I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious.  Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.

The first thing that I would like to mention is that the author seems to be incapable of spelling my name correctly.  It isn't hard, 4 whole letters but the only time that it is spelt correctly is when the text is copied from a NetBSD commit.  If you ever do read this Mr. Dickey, I am not offended, plenty of people struggle with my last name, I am quite used to it.

Before going further, just a bit of background as to how I started working on the NetBSD curses library.  I originally was using a commercial x86 port of System V UNIX on my home PC but started looking elsewhere when I found that the SL/IP implementation was bugged.  At the time Linux had no networking at all but the BSD source code had just been ported to the PC in a project known as 386BSD.  Since this had networking I picked this up and then switched to NetBSD early on.  Being used to the SYSV curses I was disappointed to find that there was no real way of handling cursor and function keys in a terminal independent manner as there is in the SYSV curses.  I developed and submitted a patch to the NetBSD curses library that added the support for the keypad() function and modified getch() to perform the conversion of key sequences into key symbols.  It was on the basis of this patch that I was offered the opportunity to be a NetBSD developer.

When working on NetBSD curses my overall philosophy is to follow The Single UNIX® Specification, Version 2: X/Open Curses, Issue 4 Version 2 
(I will refer to this as SUSv2 from now on) where possible, if a behaviour is undefined then I look to ncurses and match the behaviour there if appropriate on the basis that this normally results in less issues for someone porting an application.  I add ncurses extensions in response to problem reports (PRs).

Let's now have a look at Mr. Dickey's comments (I am tempted to use the word diatribe but that seems a bit strong...).  I won't go over all of them but will pick up the topics that he has mentioned going down his page.

I would like to pick up on the comments about making the window structure opaque.  I don't apologise for this, I believe that the fact that WINDOW * was available for applications to poke at is a violation of software layering and just wrong.  Later in Mr. Dickey's page he takes NetBSD to task over error checking (more on this later) but by making the contents of WINDOW available for the application there is little use in error checking if the application can just write what they want anyway.  It took ncurses until 2007 to opaque WINDOW.  Mr. Dickey makes some ominous sounding statements about adding functions to compensate for the opaquing.  As far as I am concerned the interface to update a window is specified by SUSv2, if there are extensions that need to be added they will be added as they are needed.
Given I made WINDOW opaque nearly 20 years ago, I have yet to see a PR to indicate that more is needed.  The positives of making WINDOW opaque are not only enforcing correct layering of software but it also means that the contents of WINDOW can be modified without requiring a major bump of the library since nothing outside the curses library has access so backward compatibility is maintained.

Making the character string in scanw const just makes sense the function does not need to modify the string so it should be a constant.  The NetBSD project does not need to worry about the compiler objecting because the base compiler works with this statement.

Wide curses was a Google Summer of Code project that was done by Ruibiao Qiu.  I think that this was a great success and added some very useful functionality to NetBSD curses.  It seems that Mr. Dickey is a bit confused by this, in actual fact, he is totally wrong on a number of points:
  1. The non-spacing characters are not stored as a linked list, they are an array
  2. The interface for setting non-spacing characters is anything that uses the complex character type (cchar_t)
He seems a bit mystified by the term non-spacing character, this term comes from SUSv2 described here
The non-spacing characters are dynamically added when required, this was my design as I thought that simply having a fixed array for every character cell in the display was a waste of memory, it would be more efficient to only allocate memory as required for the non-spacing characters.

The function specification for ungetch is ambiguous in SUSv2, NetBSD will accept the output from getch on the basis that the input stream has already been processed and it makes no sense to save the last character in what could be a multi-byte sequence.

As for testing of curses, yes, this is mine.  I am happy that at least some of our curses functions are being tested to ensure that any changes made don't affect the on-screen output and that if there are changes that they are highlighted and can be analysed for correctness.  It is too easy with terminal output to have something that "looks right" but is doing the wrong thing in the background.

Now we get to something I wrote on a mailing list:
 
Re: curses: create panel from stdscr?

I stand by my comments there.  I was referring to my attempt to report what I thought was an issue with the ncurses libmenu implementation.  I no longer have the email nor the reply from Mr. Dickey but my clear recollection of the tone and attitude was that I would not be bothered raising issues about ncurses with Mr. Dickey ever again.  I wrote the libmenu and libform libraries for NetBSD using the book "Programmer's Guide: Character User Interface (FMLI and ETI)", this is UNIX System Laboratories book so I regard it as definitive for the libraries.  I noticed a variance in the ncurses libmenu implementation but as I have stated my efforts of reporting that were not fruitful.  Mr. Dickey did reach out to me in late 2017 asking for the email I was referencing, his comment was that my response was vague, actually, my exact words were:
I will try and dig it up.  It was a very long time ago.  I think it was
related to some behaviours in libmenu.  I will try and dig back through
my mail history but it may take a bit of time.
Alas I never found the original email but looking at my commits around libmenu I would have sent that email in 1999 some time so I don't think that this is very surprising.  At the time of the reply I thought that perhaps there was some interest in actually addressing the issue but the actual fact is that all it was was a fishing expedition to provide more ammunition for an attack on NetBSD curses which is disappointing.

Getting back to the email quoted above, in the book I have create_panel() is documented as taking a WINDOW as parameter.  Though, internally, stdscr is treated as a window the books description of create_panel() does make it clear that WINDOW is expected to be the return from newwin() not just stdscr.

As for missing functions, they will be added as and when the developers have time.  A PR may guide us in what is needed first but given that NetBSD is a volunteer project the time that the developers can devote to adding new features can be variable.  We are aiming to have something that is useful and not necessarily a clone of ncurses.

Mr. Dickey makes a point about portability.  I myself have never been concerned about making NetBSD curses portable, this is not an aim that I have ever had.  I would much rather be adding new features and fixing bugs for NetBSD than struggling with the complexities of making the NetBSD curses library portable.  If others wish to do this and feed back fixes then they will be considered but in the main, portability is not a focus.

He also turns on some random gcc flags and then berates NetBSD curses for producing extra warnings.  It is difficult to determine what flags he has turned on to produce these warnings.  The NetBSD build system treats any warnings as errors and the build terminates, the NetBSD curses library builds with the standard NetBSD gcc options without issue.  Without knowing what options were uses and what defines were in effect the information that Mr. Dickey has presented is useless.  He is welcome to submit a PR using the public interface and it will be looked into.

He then notes the output differences in the wide character implementations.  Unfortunately he has not reached out to me (at least) with this information.  It would be useful to have the tools to test this issue and rectify it.  I will note that he used a rather old version of NetBSD, it would be interesting to see if these issues still exists.

We then have a snide little bit about the number of error returns and error checking.  He says "Error-checking in NetBSD curses is seen as a problem by its developers" which is absolutely untrue and uncalled for.  He seems to be forgetting at this by that, by his own admission, that NetBSD curses does not have all the functions that ncurses has so there should be less returns of status simply from that.  Even if this were not true, if we look at the ratio of ERR/OK returns per number of statements - using the numbers from his page - NetBSD curses has 12987 statements which translates to a ERR/OK return every 25 statements where as ncurses has 24566 statements which translates to a ERR/OK return ever 38 statements.  So NetBSD curses actually assigns status codes more often than ncurses.  Regardless, I view this metric as specious - simply counting returns is rubbish, look at this:

if ((x > maxx) || (y > maxy))
         return ERR;

VS:

if (x > maxx)
        return ERR;

if (y > maxy)
        return ERR;


Both do the same thing but we get double the ERR returns in the second one.  I am not saying this is what is happening but just pointing out the fallacy of counting ERR/OK assignments and trying to draw a conclusion from it without properly analysing the code.

He lambasts NetBSD for commenting out unsupported functions in pycurses, this is just a reflection of the NetBSD curses not having all the functions at the moment.  This is a pkgsrc thing and locally applied, pycurses is unique in that it wants all the curses functions so it can provide bindings, since NetBSD curses doesn't have them then the unsupported ones are commented out.  It would be ideal if they all existed, hopefully one day they will.

The final thing I would like to pick up on is the comment:

"Without documentation, there is no point in making bug reports."

Which is just a lame excuse, even the lack of documentation is something that can be a bug report.  There is a public interface for reporting bugs.  Not using it and then complaining on your own website is not helpful to NetBSD but then again, perhaps that is the intention after all.

April 08, 2020

NetBSD Blog Wifi renewal restarted

Back in 2018, Phil Nelson started a long needed WiFi-refresh, basically syncing our src/sys/net80211/ with the version from FreeBSD. He got a few things working, but then ran out of time and was unable to spend enough time on this to complete the task. I am now taking over this work, Phil hopes to join in again later this summer.

The main idea is to get better SMP (locking) support, support for newer (faster) WiFi standards, and support for virtual access points, while also making future updates (and import of drivers) from FreeBSD easier.

I have collected quite a few WiFi cards and USB dongles, but will ask for help with other hardware I don't have handy later.

I hope to have a working setup and a few working drivers by the end of this months.

Thanks to the NetBSD Foundation for funding this work! Please note that we are looking for donations again, see Fundraising 2020.

P.S.: for the curious: the first drivers I hope to have working are the ones we have already drivers for and I have hardware:

followed by things other developers can convert and have hardware (with me assisting where needed), optionally followed by some of the ones where I have hardware but we have no driver yet:


April 06, 2020

NetBSD.org pkgsrc-2020Q1 released

April 04, 2020

NetBSD Blog LLDB work concluded

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support, porting to i386.

March 2020 was the last month of my contract. During it my primary focus was to prepare integration of LLDB into NetBSD's src tree.

LLDB integration

The last important goal for the contract was to include LLDB in the NetBSD src tree. This mainly involved porting LLDB build into NetBSD src tree Makefiles. The resulting patches were sent to the tech-toolchain mailing list: [PATCH 0/7] LLDB import to src.

My proposed integration is based on LLDB tree from 2019-10-29. This matches the LLVM/Clang version currently imported in NetBSD. Newer version can not be used directly due to API incompatibility between the projects, and it is easier to backport LLDB fixes than to fix LLVM API missync.

The backports applied on top of this commit include all my contracted work, plus Kamil Rytarowski's work on LLDB. This also includes necessary fixes to make LLDB build against current NetBSD ptrace() API. Two source files in liblldbHost are renamed to ensure unique filenames within that library, as necessary to build from NetBSD Makefiles without resorting to ugly hacks.

Upstream uses to build individual LLDB components and plugins into split static libraries, then combine them all into a shared liblldb.so library. Both lldb and lldb-server executables link to it. We currently can not follow this model as LLVM and Clang sources are built without -fPIC and therefore are not suitable for shared libraries.

Therefore, we build everything as static libraries instead. This causes the logic that upstream uses to find lldb-server to fail, as it relies on obtaining the library path from the dynamic loader and finding executables relative to it. I have replaced it with hardcoded path to make LLDB work.

The patches are currently waiting for Joerg Sonnenberger to finish LLVM/Clang update that's in progress already.

Pending tasks

The exact list of pending tasks from my contract follows:

  1. Add support to backtrace through signal trampoline and extend the support to libexecinfo, unwind implementations (LLVM, nongnu). Examine adding CFI support to interfaces that need it to provide more stable backtraces (both kernel and userland).

  2. Add support for aarch64 target.

  3. Stabilize LLDB and address breaking tests from the test suite.

Notes on backtracing through signal trampoline

I have described the problem of backtracing through signal trampoline in February's report. I haven't managed to finish the work on the topic within the contract but I will try to work on it in my free time.

Most likely, the solution would involve modifying the assembly in lib/libc/arch/*/sys/__sigtramp2.S. As suggested by Andrew Cagney, the CFI directives for amd64 would look like:

NENTRY(__sigtramp_siginfo_2)
    .cfi_startproc
    .cfi_signal_frame
    .cfi_def_cfa r15, 0
    /* offsets from mcontext_t */
    .cfi_offset rax, 0x70
    .cfi_offset rbx, 0x68
    .cfi_offset rcx, 0x18
    .cfi_offset rdx, 0x10
    /* ... */
    .cfi_def_cfa rsp, 8
    movq    %r15,%rdi
    movq    $SYS_setcontext, %rax
    syscall
    movq    $-1,%rdi /* if we return here, something is wrong */
    movq    $SYS_exit, %rax
    syscall
    .cfi_endproc
END(__sigtramp_siginfo_2)

Addressing breaking tests

While the most important functions of LLDB work on NetBSD, there are still many test failures. At this moment, there are 80 instances of @expectedFailureNetBSD decorator and 18 cases of @skipIfNetBSD. The former generally indicates that the test reliably fails on NetBSD and needs a fix, the latter is sometimes used to decorate tests specific to other systems but also to indicate that the test crashes, hangs or otherwise can not be reliably run.

Some tests are failing due to the concurrent signal kernel bug explained in the previous post and covered by XFAIL-ing ATF tests.

New regressions both in LLDB and in LLVM in general appear every month. Most of them are fixed by their authors once we report them. I will continue fighting new bugs in my free time and trying to keep the build bot green.

This work is sponsored by The NetBSD Foundation

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

https://netbsd.org/donations/#how-to-donate


April 02, 2020

NetBSD.org NetBSD 8.2 released
NetBSD.org Extended support of the netbsd-7 branch

April 01, 2020

NetBSD.org New Developer in March 2020

March 20, 2020

Unix Stack Exchange How to change the font size in NetBSD (TTY no GUI)

NetBSD 7.1 (GENERIC.201703111743Z) amd64

In TTY without gui , the better screen resolution is set at boot time but the font is really to small.
So how to change that

On debian dpkg-reconfigure console-setup
in grub GRUB_GFXMODE=800x600

I find wsconscfg with a example :

wsconscfg -t 80x50 -e vt100 3

but I get

wsconscfg: WSDISPLAYIO_ADDSCREEN: Device not configured

I look at man wsconscfg wsconsctl wscons wsdisplay vga etc..
All of them link to the other but I can't find a way.


March 17, 2020

Super User Most Windows-like text editor that can be used through PuTTY in NetBSD?

Please help me find an editor that behaves more like the modern Windows editors do, that I could use on a NetBSD box through PuTTY.

This includes the absense of the distinction between view/insert/append modes, basic shortcuts such as Ctrl+Left/Right to jump a word at a time, and other niceties such as perhaps Home/End and PageUp/Down moving the cursor around (in my vi they just ding).

I can live without macros because I only ever tweak an occasional setting this way, and nothing else.

pico is pretty close, but I was wondering if there are other editors that might fit this bill?


March 15, 2020

Unix Stack Exchange pkgin installation problem (NetBSD)

I just installed NetBSD 7.1.1 (i386) on my old laptop.

During the installation, I could not install pgkin (I don't know why), so I skipped it and now I have a NetBSD 7.1.1 installed on my laptop without pkgin.

My problem is "How to install pkgin on NetBSD (i386) ?"

I found this Unixmen tutorial and I followed it:

I tried :

#export PKG_PATH="http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/6.0_BETA_current/All/"
# pkg_add -v pkgin

And I got :

pkg_add: Can't process ftp://ftp.netbsd.org:80/pub/pkgsrc/packages/NetBSD/amd64/6.0_BETA_current/All/%0d/pkgin*: Not Found
pkg_add: no pkg found for 'pkgin',sorry.
pkg_add: 1 package addition failed

I know this is a wrong command because this ftp address is for amd64 while my laptop and this NetBSD is i386. (I can't find the correct command for i386 )

I also followed instructions of pkgin.net, and I did

git clone https://github.com/NetBSDfr/pkgin.git

on another computer and copied the output (which is a folder name pkgin) to my NetBSD (my NetBSD doesn't have 'git' command)

and then I did :

./configure --prefix=/usr/pkg --with-libraries=/usr/pkg/lib --with-includes=/usr/pkg/include

and then :

make

but I got :

#   compile  pkgin/summary.o
gcc -O2    -std=gnu99    -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-sign-compare  -Wno-traditional  -Wa,--fatal-warnings -Wreturn-type -Wswitch -Wshadow -Werror    -DPKGIN_VERSION=\""0.9.4 for NetBSD-7.1.1 i386"\" -DNETBSD  -g -DLOCALBASE=\"/usr/local\"           -DPKG_SYSCONFDIR=\"/usr/local/etc\"         -DPKG_DBDIR="\"/var/db/pkg\""           -DDEF_LOG_DIR="\"/var/db/pkg\""         -DPKGIN_DB=\"/var/db/pkgin\"            -DPKGTOOLS=\"/usr/local/sbin\" -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_LARGE_FILES -DCHECK_MACHINE_ARCH=\"i386\" -Iexternal -I. -I/usr/local/include  -c    summary.c
*** Error code 1

Stop.
make: stopped in /root/pkgin

I think this error occurs because of the dependencies. (which is mentioned in pkgin.net) but still, don't know how to install those dependencies.

EDIT: I found "http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/7.1.1/All/" but it still says

no pkg fond for 'pkgin', sorry

SOLVED:

** I solved the problem by writing 7.1 instead of 7.1.1**


March 06, 2020

Stack Overflow How do I set root password for mysql 5.7 on netbsd

I'm trying to get mySQL 5.7 working on NetBSD (unix). When I try "mysql -u root" I get

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

When I try "mysqld --initialize-insecure" I get

[ERROR] --initialize specified but the data directory has files in it.

What do I do?


March 02, 2020

Emile Heitor Let's Encrypt certificates using LEGO

This post is more like a self-reminder on how I setup automatic SSL/TLS certificate renewal on my servers.

I chose LEGO to handle my certificates renewal with Let’s Encrypt because it’s simple to use, has no dependency, great documentation and is worked on at a constant pace.

I found this and this articles very useful, but they are outdated in their use of the tls and http parameters. So here are my notes.

This procedure is Debian GNU/Linux based but I also used it pretty much as-is on NetBSD and FreeBSD, only nginx related PATHs changed.

nginx

In /etc/nginx/sites-available/default:

1
include /etc/nginx/sites-available/letsencrypt;

Which contains:

1
2
3
4
location /.well-known/acme-challenge {
proxy_pass http://127.0.0.1:8181;
proxy_set_header Host $host;
}

Check nginx.conf syntax then reload it.

1
2
$ sudo nginx -t
$ sudo nginx -s reload

Let’s Encrypt

Go to a writable directory where LEGO will write the challenges

1
$ cd ~/www/letsencrypt

To request and install a new certificate, first prepare nginx.conf:

/etc/nginx/sites-available/https:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
listen443 ssl;
listen[::]:443 ssl;
server_namekorriban.imil.net;

ssl_certificate/usr/local/etc/letsencrypt/certificates/korriban.imil.net.crt;
ssl_certificate_key/usr/local/etc/letsencrypt/certificates/korriban.imil.net.key;

# ...

# for future use of the tls challenge
location /.well-known/acme-challenge {
proxy_pass http://127.0.0.1:4444;
proxy_set_header Host $host;
}

# ...

Then execute LEGO with desired parameters:

1
$ sudo lego --email="[email protected]" --domains="korriban.imil.net" --http.port :8181 --http --path=/usr/local/etc/letsencrypt run

And check / reload nginx.

If you need a multi-domain certificate, simply add multiples --domains.

To renew the certificate using the tls challenge if it expires within 30 days:

1
$ sudo /usr/local/bin/lego --email="[email protected]" --domains="korriban.imil.net" --tls.port :4444 --tls --path=/usr/local/etc/letsencrypt renew --days 30

Automatic renewal

1
$ cat bin/lerenew.sh
1
2
3
4
5
#!/bin/sh

cd /home/imil/www/letsencrypt
/usr/local/bin/lego --email="[email protected]" --domains="korriban.imil.net" --tls.port :4444 --tls --path=/usr/local/etc/letsencrypt renew --days 30
nginx -s reload

cron

1
00 3 * * * /home/imil/bin/lerenew.sh >/home/imil/log/lerenew.log 2>&1

reload mail system

My mail system also uses Let’s Encrypt certificates, it is hosted on virtual machines in the same subnet as the web server, which exports /usr/local/etc/letsencrypt/ via NFS. The mail server runs NetBSD. In order to watch any modifications in the certificates directory, I use direvent, which will call a script to reload both dovecot and postfix:

1
$ cat /usr/pkg/etc/direvent.conf
1
2
3
4
5
6
7
8
9
10
11
syslog {
facility local0;
tag "direvent";
print-priority yes;
}

watcher {
path /usr/local/etc/letsencrypt/certificates;
event write;
command "/home/imil/bin/mailrestart.sh";
}
1
$ cat bin/mailrestart.sh
1
2
3
4
5
6
#!/bin/sh

/usr/sbin/postfix reload
/usr/pkg/bin/doveadm reload

su imil -c 'echo "done"|mail -s "postfix and dovecot reloaded" imil'

direvent is started from /etc/rc.local as there’s no init script given with the package.

1
/usr/pkg/bin/direvent /usr/pkg/etc/direvent.conf

February 17, 2020

OS News NetBSD 9.0 released

The NetBSD Project is pleased to announce NetBSD 9.0, the seventeenth major release of the NetBSD operating system. This release brings significant improvements in terms of hardware support, quality assurance, security, along with new features and hundreds of bug fixes.

Support for the ARM architecture seems to be a major pillar of this new release.


February 13, 2020

Unix Stack Exchange Difference between `pkgin` and `pkg_*`, and which to use?

Coming from Linux land, I assumed that pkgin is some sort of higher-level frontend for pkg_add et al, something like what apt is to dpkg or yum/dnf to rpm. But pkg_add appears to handle installation from network, dependencies, as well as automatic updates (the things that on Linux would be the responsibility of the front-ends rather than the underlying package tool), so now the roles of pkgin and (what I gather are the more traditional) pkg_ tools seem a bit unclear to me. The only difference that I'm presently aware of is that pkgin does not handle installation from sources.


February 03, 2020

Kimmo Suominen Updated sudo

I updated security/sudo to 1.8.31. A fix for CVE-2019-18634 is included.

Originally only versions before 1.8.26 were thought to be vulnerable. Later analysis, however, showed that versions 1.8.26 – 1.8.30 are also vulnerable.

Here’s what’s new:


January 27, 2020

Roy Marples openresolv-3.10.0 released

with the following changes:

ftp://roy.marples.name/pub/openresolv/openresolv-3.10.0.tar.xz
https://roy.marples.name/downloads/openresolv/openresolv-3.10.0.tar.xz

Roy Marples dhcpcd-8.1.6 released

with the following changes:

ftp://roy.marples.name/pub/dhcpcd/dhcpcd-8.1.6.tar.xz
http://roy.marples.name/downloads/dhcpcd/dhcpcd-8.1.6.tar.xz

This should be the last dhcpcd-8 release before dhcpcd-9 :)


January 25, 2020

Stack Overflow Alternatives to Vagrant for OS emulation/development [closed]

I am the maintainer psutil project. Since psutil supports different operating systems, I've been using Vagrant in order to to emulate them and develop on them from my Linux box. This worked flawlessly for many years (due to Vagrantfiles, which are great), but unfortunately I got to a point that Vagrant is no longer usable anymore. Problems derive mostly from the inability to mount shared folders, freezes when connecting via SSH and occasional incompatibilities between Vagrant and VirtualBox after a system upgrade. Are there viable alternatives to Vagrant? My working environment is Ubuntu and the systems I need to emulate are BSD*, MacOS and OpenSolaris (for Windows I use VirtualBox directly).


January 24, 2020

Kimmo Suominen Added patches to libxml2

I added patches to textproc/libxml2 from an upstream commit and an upstream pull request to address CVE-2020-7595 and CVE-2019-20388 respectively. Version 2.9.10nb1 includes the patches.


January 19, 2020

Kimmo Suominen Patch for HTTP Request Smuggling

I added a patch to www/nginx from an upstream commit to address CVE-2019-20372. Version 1.16.1nb2 includes the patch.


January 15, 2020

Roy Marples Anonymity Profiles for DHCP Clients aka RFC 7844

DHCP clients by default send a fair chunk of data which can identify you to the local DHCP server. In return they provide you with a stable IP address and configuration parameters.

At a bare minimum, the hardware address of the interface is sent - this is required to work.

So, how to solve this dilema of wanting total anonymity? The answer is to randomise the hardware address. This will happen when the carrier is down OR dhcpcd starts with the interface down. Then, dhcpcd will use this random hardware address to set a DUID LL which will be used inplace of any saved DUID and set the IAID to all zeros. This combo is used by DHCP and DHCPv6 to identify a lease. As this is randomised each time the carrier comes up you get a different IP address!

Try not to use this on an unstable link as it could drain the DHCP server of addresses :(

But we can't stop there! dhcpcd also sends some identifying options as well! For example, this is sent in the vendor class identifier:
dhcpcd-8.99.0:NetBSD-9.99.17:amd64:x86_64

It does not identify you or the device in anyway, but it does say what software is being used on which hardware. This could be used by DHCP servers to hand out a specific image to download and boot from TFTP for network boot clients.

Now, there are a gazzillion and one DHCP options out there - we don't know what you've configured. So dhcpcd will mask all of them when anonymous mode is activated, unless they are essential for enabling dhcpcd to work correctly on the network. But wait! What if you really want to leak something? Like say your on a corporate network that uses DHCP security and still want to remain anonymous? Well you can! Any request or option after the anonymous option in dhcpcd.conf is turned on. So the placing of the anonymous directive is important, unlike other dhcpcd options. So far this is the only implementation of RFC 7844 which does this :)

This is NOT enabled by default because most people want stable addresses AND a flappy link could drain addresses as disussed earlier.


January 11, 2020

Kimmo Suominen Cherry-picked patches for ncurses

In order to reduce the number of vulnerabilities on my systems, I added some patches to devel/ncurses (and devel/ncursesw) to address CVE-2018-19211, CVE-2019-17594, and CVE-2019-17595. Version 6.1nb7 includes the patches.


January 09, 2020

Roy Marples structure padding in C

Whilst developing Privilege Separation in dhcpcd, I had to come up with an IPC design for it. Of course, that involves creating structures.

So far, my structures in dhcpcd are long lived - or rather the scope is design to live outside of where it was created. As such they are created on the heap and are at the mercy of malloc. Generally I use calloc so that the whole area is inited to zero as uninitialised memory is bad.

So I decided to start out and see if I can just create the structures I need on the stack. Turns out I could! Yay! Now, how to you initialise a structure on the stack to all zeros? First let us consider this structure:

struct ps_addr {
    sa_family_t psa_family;
    union {
        struct in_addr psau_in_addr;
        struct in6_addr psau_in6_addr;
    } psa_u;
#define psa_in_addr     psa_u.psau_in_addr
#define psa_in6_addr    psa_u.psau_in6_addr
};

The first way is memset:

struct ps_addr psa;

memset(&psa, 0, sizeof(psa));
psa.psa_family = AF_INET;

But what if you could avoid memset? Luckily the C standard allows setting any member and will zero all other members. So we can do this:

struct ps_addr psa = { .psa_family = AF_INET };

Wow!!! So simple. This reduces binary size a fair bit. But then I turned on the Memory Sanitiser and boom, it crashed hard. Why?

The answer is simple - padding. Eric S Raymond gives a very good writeup about the problem. Basically, the standard will initialise any unintialised members to zero - but padding added for alignent isn't a member! So we need to ensure that our structure requires zero padding.

Here is the new struct:

struct ps_addr {
    sa_family_t psa_family;
    uint8_t psa_pad[4 - sizeof(sa_family_t)];
    union {
        struct in_addr psau_in_addr;
        struct in6_addr psau_in6_addr;
    } psa_u;
#define psa_in_addr     psa_u.psau_in_addr
#define psa_in6_addr    psa_u.psau_in6_addr
};

And it allows the former structure initialisation to work and memory sanitisers are happy - so happy days :) Now, if anyone can tell me what I can use instead of the magic number 4 in the above I'd be even happier!


January 08, 2020

Amitai Schlair NYCBUG: What is notqmail?

On Wednesday, January 8, I attended the New York City BSD User Group to present What is notqmail?, a perhaps not entirely surprising followup to my March talk. At the time, I’d been trying to avoid creating yet another qmail fork. This talk is about my failure — notqmail is alive and well — and about our success thus far.

Abstract:

We all use email, so we all use email servers. notqmail is software for running an email server. Someday, if we do a good job, some of the many articles about how and why to run your own will recommend notqmail.

notqmail is a community-driven fork of qmail, beginning where netqmail left off: providing stable, compatible, small releases to which existing qmail users can safely update. notqmail also aims higher: developing an extensible, easily packaged, and increasingly useful modern mail server.


Work with me

Would you personally benefit from an individualized session with an experienced, inquisitive, and empathetic conversation partner? Maybe you’re facing a challenging situation at work, a learning opportunity in some code — or both. Last week a new client went from “frustrated” to “energized” in the span of an hour. Get in touch: latentagility.com

Would your org benefit from a rare combination of technical coaching and impactful conversations? (Take other people’s word for it, not mine.) It’s not too late to book some time with me in 2020. Let’s talk about what fits for you.


January 06, 2020

Unix Stack Exchange Is there anything similar to CrystalDiskMark for UNIX?

If you're shopping for an SSD, you've surely seen one of those screenshots from CrystalDiskMark with a few green squares, and the 2x4 matrix with results of doing read/write tests for the given hardware:

However, I've never seen anything similar for UNIX — at most what you get is a dd for sequential read/write tests, which gives no indication on the IOPS parameters of the hardware in question.

Is there anything similar to CrystalDiskMark for UNIX to perform various read/write tests like 4KiB Q8T8 etc?

I searched and found the following items in OpenBSD ports, but they seem rather stale (to say the least — randread is still hosted on SourceForge in 2020 and BYTE magazine has reportedly ceased online publication in 2013), and none of these tools make any mention of evaluating modern SSD performance, for which you probably have to have some sort of extra code to deal with the IO queues and threads or whatnot:


January 03, 2020

Roy Marples dhcpcd-8.1.5 released

with the following changes:

If you are suffering from IPv6 addresses not transitioning from the tentative state (regression from dhcpcd-8.1 on Linux) you will need to do one of the following after installing dhcpcd:

ftp://roy.marples.name/pub/dhcpcd/dhcpcd-8.1.5.tar.xz
http://roy.marples.name/downloads/dhcpcd/dhcpcd-8.1.5.tar.xz


December 21, 2019

Stack Overflow Rump unikernel compile successfully on a server but error on Ubuntu 19.04

I downloaded rumprun and followed the Tutorial to compile and install it.

git submodule update --init
CC=cc ./build-rr.sh hw

I have compiled successfully on a Ubuntu 18.04 system. When I copied the same code on that system to my system Ubuntu 19.04 to compile. It failed to compile. There were many error messages. Many of them are some warning. For example,

rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../sys/conf.h:129:19: error: cast between incompatible function types from 'int (*)(void)' to 'int (*)(dev_t,  int,  int,  struct lwp *)' {aka 'int (*)(long unsigned int,  int,  int,  struct lwp *)'} [-Werror=cast-function-type]
 #define noclose  ((dev_type_close((*)))enodev)
rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../net/bpf.c:184:13: note: in expansion of macro 'noclose'
  .d_close = noclose,
             ^~~~~~~
rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../sys/conf.h:130:18: error: cast between incompatible function types from 'int (*)(void)' to 'int (*)(dev_t,  struct uio *, int)' {aka 'int (*)(long unsigned int,  struct uio *, int)'} [-Werror=cast-function-type]
 #define noread  ((dev_type_read((*)))enodev)
rumprun/src-netbsd/sys/rump/dev/lib/libbpf/../../../../net/bpf.c:185:12: note: in expansion of macro 'noread'
  .d_read = noread,
            ^~~~~~

I searched for these compiling errors but I haven't found the same bugs. I don't know why this happens. How can I do to solve this problem? Thanks for your help!


December 13, 2019

Super User NetBSD - no pkg

After full installation of latest NetBSD I'm tried to launch pkgin, but received pkgin not found, also I've got same for pkgsrc. Then I've found, that there's no /usr/pkg location.

That's normal or I've did something wrong?


December 01, 2019

Server Fault ssh tunnel refusing connections with "channel 2: open failed"

All of a sudden (read: without changing any parameters) my netbsd virtualmachine started acting oddly. The symptoms concern ssh tunneling.

From my laptop I launch:

$ ssh -L 7000:localhost:7000 [email protected] -N -v

Then, in another shell:

$ irssi -c localhost -p 7000

The ssh debug says:

debug1: Connection to port 7000 forwarding to localhost port 7000 requested.
debug1: channel 2: new [direct-tcpip]
channel 2: open failed: connect failed: Connection refused
debug1: channel 2: free: direct-tcpip: listening port 7000 for localhost port 7000, connect from 127.0.0.1 port 53954, nchannels 3

I tried also with localhost:80 to connect to the (remote) web server, with identical results.

The remote host runs NetBSD:

bash-4.2# uname -a
NetBSD host 5.1_STABLE NetBSD 5.1_STABLE (XEN3PAE_DOMU) #6: Fri Nov  4 16:56:31 MET 2011  [email protected]:/m/obj/m/src/sys/arch/i386/compile/XEN3PAE_DOMU i386

I am a bit lost. I tried running tcpdump on the remote host, and I spotted these 'bad chksum':

09:25:55.823849 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 67, bad cksum 0 (->3cb3)!) 127.0.0.1.54381 > 127.0.0.1.7000: P, cksum 0xfe37 (incorrect (-> 0xa801), 1622402406:1622402421(15) ack 1635127887 win 4096 <nop,nop,timestamp 5002727 5002603>

I tried restarting the ssh daemon to no avail. I haven't rebooted yet - perhaps somebody here can suggest other diagnostics. I think it might either be the virtual network card driver, or somebody rooted our ssh.

Ideas..?


November 25, 2019

Super User What device does NetBSD use for a USB modem?

I'm testing some software on NetBSD 8.1 x86_64. The software opens a USB modem and issues AT commands. The software tested OK on Debian, Fedora, OS X, and OpenBSD. The software is having trouble on NetBSD.

NetBSD's dmesg shows:

umodem0 at uhub1 port 1 configuration 2 interface 0
umodem0: U.S.Robotics (0xbaf) USB Modem (0x303), rev 2.00/2.00, addr 2, iclass 2/2
umodem0: data interface 1, has CM over data, has break
umodem0: status change notification available
ucom0 at umodem0

If I am parsing the NetBSD man pages properly (which may not be the case), I should be able to access the modem via /dev/ucom0. Also see UMODEM(4) man page.

The test user is part of the dialer group. The software was not able to open /dev/ucom0, /dev/umodem0, ucom0 or umodem0. All open's result in No such file or directory. Additionally, there are no /dev/ttyACMn or /dev/cuaUn devices.

How do I access the modem on NetBSD?


November 17, 2019

Stack Overflow Compile only kernel module on NetBSD

Is there a way to compile only a kernel module on NetBSD? I can't seem to figure out how to do so without recompiling the entire kernel. Thanks!


October 25, 2019

Stack Overflow How can I make a NetBSD VM halt itself in google compute engine

I've got a batch job that I want to run in google compute engine on a NetBSD instance. I expected that I could just shutdown -hp now at the end of the job and the instance would be powered off. But when I do that it still remains in the running state according to the google cloud console and CLI. How do I make a NetBSD virtual machine in google cloud shut itself off when it's no longer needed?

Note: Google cloud SDK is not available on NetBSD


August 20, 2019

Super User How to run a Windowed JAR file over SSH without installing JRE and without root access on NetBsd?

First, I can use Java, but for what I want to achieve (building a database where othe only application supporting the format is in Java), I need 100Gb of RAM during 20 hours.

I have access to a server with the required RAM, but not as root and no JRE is available. The same is true for the Xorg libraries.

Here’s the uname :

8.0_STABLE NetBSD 8.0_STABLE (GENERIC) #0: Sun Feb 24 10:50:49 UTC 2019  [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC amd64

The Linux layer is installed, but nothing else is installed : not even Glibc, so the only applications which can be run are the ones which are statically compiled.

So not only Java isn’t Installed, but some of the require shared libraries are missing…
However, I have full write access to my $HOME directory, and I can run my own executables from there.

Is a way to convert a Jar file into a NetBsd Executable or Linux statically linked executable ? I have also the source code of the Jar file if compiling is an acceptable solution.

I only found about ɢᴄᴊ, but I’m unsure if Java 7 is supported…

Amitai Schlair Announcing notqmail

Running my own email server has its challenges. Chief among them: my favorite mail-server software hasn’t been updated since I started using it in 1998.

The qmail logo
qmail logo

Okay, that’s not entirely true. While qmail hasn’t been updated by its original author, a group of respected users created netqmail, a series of tiny updates that were informed, conservative, and careful. By their design, it was safe for everyone running qmail to follow netqmail, so everyone did. But larger changes in the world of email — authentication, encryption, and ever-shifting anti-spam techniques — remained as puzzles for each qmail administrator to solve in their own way. And netqmail hasn’t been updated since 2007.

One fork per person

In the interim, devotees have continued maintaining their own individual qmail forks. Some have shared theirs publicly. I’ve preferred the design constraints of making minimal, purpose-specific, and conflict-avoidant add-ons and patches. Then again, these choices are motivated by the needs of my qmail packaging, which I suppose is itself a de facto fork.

I’ve found this solo work quite satisfying. I’ve learned more C, reduced build-time complexity, added run-time configurability, and published unusually polished and featureful qmail packages for over 20 platforms. Based on these experiences, I’ve given dozens of workshops and talks. In seeking to simplify system administration for myself and others, I’ve become a better programmer and consultant.

Still, wouldn’t it be more satisfying if we could somehow pool our efforts? If, long after the end of DJB’s brilliant one-man show, a handful of us could shift how we relate to this codebase — and to each other — in order to bring a collaborative open-source effort to life? If, with netqmail as inspiration, we could produce safe updates while also evolving qmail to meet more present-day needs?

One fork per community

My subtle artwork
notqmail logo == qmail logo overlaid by a red circle with a slash through it

Say hello to notqmail.

Our first release is informed, conservative, and careful — but bold. It reflects our brand-new team’s rapid convergence on where we’re going and how we’ll get there. In the span of a few weeks, we’ve:

I say “bold” because, for all the ways we intend to hew to qmail tradition, one of our explicit goals is a significant departure. Back in the day, qmail’s lack of license, redistribution restrictions, technical barriers, and social norms made it hard for OS integrators to create packages, and hard for package users to get help. netqmail 1.06 expressed a desire to change this. In notqmail 1.07, we’ve made packaging much easier. (I’ve already updated pkgsrc from netqmail to notqmail, and some of my colleagues have prepared notqmail RPM and .deb packages.) Further improvements for packagers are part of what’s slated for 1.08.

What’s next

Looking much further ahead, another of our explicit goals is “Meeting all common needs with OS-provided packages”. We have a long way to go. But we couldn’t be off to a better start.

By our design, we believe we’ve made it safe for everyone running qmail to follow notqmail. We hope you’ll vet our changes carefully, then update your installations to notqmail 1.07. We hope you’ll start observing us as we continue the work. We hope you’ll discuss freely on the qmail mailing list. We hope you’ll be a part of the qmail revival in ways that are comfortable for you. And we hope that, in the course of time, notqmail will prove to be the community-driven open-source successor to qmail.


July 13, 2019

Jeremy C. Reed 2019-July-13 pfSense Essentials Book Writing

This week I received my printed proof from the printer and enabled it to be printed. It is now for sale at Amazon and Barnes and Noble,

I set an alarm to work on it very early a few days a week and it took me a few years. (I am blessed to only commute a few times a year, so I make sure that I don't waste that gifted time.)

This book was written using Docbook using NetBSD and vi. The print-ready book was generated with Dblatex version 0.3.10 with a custom stylesheet, pdfTeX 3.14159265-2.6-1.40.19 (Web2C 2018), and the TeX document production system installed via Tex Live and Pkgsrc. Several scripts and templates were created to help have a consistent document.

The book work was managed using the Subversion version control software. I carefully outlined my steps in utilizing the useful interfaces and identified every web and console interface. The basic writing process included adding over 350 special comment tags in the docbook source files that identified topics to cover and for every pfSense web interface PHP script (highlighting if they were main webpages from the pfSense menu). As content was written, I updated these special comments with a current status. A periodic script checked the docbook files and the generated book and reported on writing progress and current needs.

During this writing, nearly every interface was tested. In addition, code and configurations were often temporarily customized to simulate various pfSense behaviors and system situations. Most of the pfSense interface and low-level source code was studied, which helped with identifying pfSense configurations and features that didn't display in standard setups and all of its options. The software was upgraded several times and installed and ran in multiple VMs and hardware environments with many wireless and network cards, including with IPv6. In addition, third-party documentation and even source code was researched to help explain pfSense configurations and behaviors.

As part of this effort, I documented 352 bugs (some minor and some significant) and code suggestions that I found from code reading or from actual use of the system. (I need to post that.)

The first subversion commit for this book was in July 2014. It has commits in 39 different months with 656 commits total. The book's docbook source had 3789 non-printed comments and 56,193 non-blank lines of text. The generated book has over 180,000 words. My subversion logs show I have commits on 41 different Saturdays. Just re-reading with cleanup took me approximately 160 hours.


July 08, 2019

Server Fault Webserver farm with NFS share (autofs failure)

I am trying to set up the farm of webservers, consisting of the internal, external and worker servers.

  1. The actual sites content is stored on internal NFS server deep in internal network. All sites contents management is centralized.

  2. BSD-based external servers have Lighttpd doing all the HTTP/HTTPS job, serving static content. Main NFS share is auto-mounted via special path, like /net/server/export/www/site/ (via amd).

  3. Every Lighttpd have fastcgi parameters pointing to several worker servers, which have php-fpm working (for example). Different sites may require different php versions or arrangement, so www01 and www02 may serve site "A" having php-fpm over PHP 5.6 and www05 and www06 will serve site "B" having php-fpm over PHP 7.2.

  4. Every worker get requests for certain sites (one or more) with path /net/server/export/www/site and execute PHP or any other code. They also have amd (for BSD) and autofs (for Linux) working.

  5. For some sites Lighttpd may not forward fastcgi, but do proxying instead, so workers can have Apache or other web-server (even Java-based) working.

External servers are always BSD, internal servers too, but workers can be different upon actual needs.

This all work good when workers are BSD. If we are using Linux on workers - it stops working when share is automatically unmounted. When one tries to access the site he will get error 404. When I connect to server via ssh I will see no mounted share on "df -h". If I do any "ls" on /net/server/export - it is self-mounted as intended and site starts to work. On BSD-systems df show amd shares always mounted despite of 60 seconds dismount period.

I believe there is a difference between amd and autofs approach, php-fpm calls on Linux become some kind of "invisible" to autofs and do not cause auto-mount, because any other access to /net/server/ work at any time and do cause auto-mount. Also, this happens not with php-fpm only, Apache serving static content on auto-mounted NFS share behave same way.

Sorry for long description, but I tried to describe it good. The main question here - is anyone know why calls to /net/server may not cause auto-mount in autofs and how to prevent this behavior.

For lot of reasons I do not consider using static mounting, so this is not an option here. As for Linux versions - mostly it was tested on OEL 7.recent.


July 03, 2019

Super User Using a Console-only NetBSD VM

I am experimenting with NetBSD and seeing if I can get the Fenrir screenreader to run on it. However, I hit a snag post install; the console that I was using for the installation was working perfectly fine, however it stopped working alltogether once I completed the install. For reference, here is the line I used for virt-install:

virt-install --connect qemu:///system -n netbsd-testing \
             --ram 4096 --vcpus=8 \
             --cpu=host \
             -c /home/sektor/Downloads/boot-com.iso  \
             --os-type=netbsd --os-variant=netbsd8.0 \
             --disk=pool=devel,size=100,format=qcow2 \
             -w network=default --nographics 

When it asked me for the type of terminal I was using (this being the NetBSD install program), I accepted the default which was VT200. As I recall, I told it to use the BIOS for booting, and not any of the comm serial ports. Has anyone had any further experience with using no graphics on a Libvirt virtualized machine, and have any points as to how to get a working console?

Thanks.


March 07, 2019

Amitai Schlair NYCBUG: Maintaining qmail in 2019

On Wednesday, March 6, I attended New York City BSD User Group and presented Maintaining qmail in 2019. This one pairs nicely with my recent DevOpsDays Ignite talk about why and how to Run Your @wn Email Server! That this particular “how” could be explained in 5 minutes is remarkable, if I may say so myself. In this NYCBUG talk — my first since 2014 — I show my work. It’s a real-world, open-source tale of methodically, incrementally reducing complexity in order to afford added functionality.

My abstract:

qmail 1.03 was notoriously bothersome to deploy. Twenty years later, for common use cases, I’ve finally made it pretty easy. If you want to try it out, I’ll help! (Don’t worry, it’s even easier to uninstall.) Or just listen as I share the sequence of stepwise improvements from then to now — including pkgsrc packaging, new code, and testing on lots of platforms — as well as the reasons I keep finding this project worthwhile.

Here’s the video:


January 25, 2019

Amitai Schlair DevOpsDays NYC: Run Your @wn Email Server!

In late January, I was at DevOpsDays NYC in midtown Manhattan to present Run Your @wn Email Server!

My abstract:

When we’re responsible for production, it can be hard to find room to learn. That’s why I run my own email server. It’s still “production” — if it stays down, that’s pretty bad — but I own all the decisions, take more risks, and have learned lots. And so can you! Come see why and how to get started.

With one command, install famously secure email software. A couple more and it’s running. A few more and it’s encrypted. Twiddle your DNS, watch the mail start coming in, and start feeling responsible for a production service in a way that web hosting can’t match.


March 17, 2018

Hubert Feyrer The adventure of rebuilding g4u from source
I was asked by a long-time g4u user on help with rebuilding g4u from sources. After pointing at the instructions on the homepage, we figured out that a few lose odds and ends didin't match. After bouncing some advices back and forth, I ventured into the frabjous joy of starting a rebuild from scratch, and quick enough ran into some problems, too.

Usually I cross-compile g4u from Mac OS X, but for the fun of it I did it on NetBSD (7.0-stable branch, amd64 architecture in VMware Fusion) this time. After waiting forever on the CVS checkout, I found that empty directories were not removed - that's what you get if you have -P in your ~/.cvsrc file.

I already had the hint that the "g4u-build" script needed a change to have "G4U_BUILD_KERNEL=true".

From there, things went almost smooth: building indicated a few files that popped up "variable may be used uninitialized" errors, and which -- thanks to -Werror -- bombed out the build. Fixing was easy, and I have no idea why that built for me on the release. I have sent a patch with the required changes to the g4u-help mailing list. (After fixing that I apparently got unsubscribed from my own support mailing list - thank you very much, Sourceforge ;)).

After those little hassles, the build worked fine, and gave me the floppy disk and ISO images that I expected:

>       ls -l `pwd`/g4u*fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u1.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u2.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u3.fs
>       -rw-r--r--  2 feyrer  staff  1474560 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u4.fs
>       ls -l `pwd`/g4u.iso
>       -rw-r--r--  2 feyrer  staff  6567936 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u.iso
>       ls -l `pwd`/g4u-kernel.gz
>       -rw-r?r--  1 feyrer  staff  6035680 Mar 17 19:27 /home/feyrer/work/NetBSD/cvs/src-g4u.v3-deOliviera/src/distrib/i386/g4u/g4u-kernel.gz 
Next steps are to confirm the above changes as working from my faithful tester, and then look into how to merge this into the build instructions .

January 04, 2018

Hubert Feyrer NetBSD 7.1.1 released
On December 22nd, NetBSD 7.1.1 was released as premature christmas present, see the release annoucement.

NetBSD 7.1.1 is the first update with security and critical fixes for the NetBSD 7.1 branch. Those include a number of fixes for security advisories, kernel and userland.

Hubert Feyrer New year, new security advisories!
So things have become a bit silent here, which is due to reallife - my apologies. Still, I'd like to wish everyone following this here a Happy New Year 2018! And with this, a few new security advisories have been published:
Hubert Feyrer 34C3 talk: Are all BSDs created equally?
I haven't seen this mentioned on the NetBSD mailing lists, and this may be of interest to some - there was a talk about security bugs in the various BSDs at the 34th Chaos Communication Congress:

In summary, many reasons for bugs are shown in many areas of the kernel (system calls, file systems, network stack, compat layer, ...), and what has happened after they were made known to the projects.

As a hint, NetBSD still has a number of Security Advisories to publish, it seems. Anyone wants to help out the security team? :-)


June 22, 2017

Server Fault How to log ssh client connection/command?

I would like to know how i could log SSH command lines a user is using on a server. For exemple, if the user Alex on my server is doing the following set of commands :

$ cd /tmp
$ touch myfile
$ ssh [email protected]
$ ssh [email protected]
$ vim anotherfile
$ ssh [email protected]

I would like to log the ssh commands used on the server in a file which looks like :

[2014-07-25 10:10:10] Alex : ssh [email protected]
[2014-07-25 10:18:20] Alex : ssh [email protected]
[2014-07-25 11:15:10] Alex : ssh [email protected]

I don't care what he did during his ssh session, i just want to know WHEN and TO WHERE he made a connection to another server.

The user is not using bash and i would like to avoid manipulating .bash_history anyway as the user can modify it.

Any clue on this ?

Thank you :)

edit : to be more specific :

a user connects to a server A and then connects from the server A to server B. I want to track down to which server he connects through ssh from server A.


June 08, 2017

Hubert Feyrer g4u 2.6 released
After a five-year period for beta-testing and updating, I have finally released g4u 2.6. With its origins in 1999, I'd like to say: Happy 18th Birthday, g4u!

About g4u: g4u ("ghosting for unix") is a NetBSD-based bootfloppy/CD-ROM that allows easy cloning of PC harddisks to deploy a common setup on a number of PCs using FTP. The floppy/CD offers two functions. The first is to upload the compressed image of a local harddisk to a FTP server, the other is to restore that image via FTP, uncompress it and write it back to disk. Network configuration is fetched via DHCP. As the harddisk is processed as an image, any filesystem and operating system can be deployed using g4u. Easy cloning of local disks as well as partitions is also supported.

The past: When I started g4u, I had the task to install a number of lab machines with a dual-boot of Windows NT and NetBSD. The hype was about Microsoft's "Zero Administration Kit" (ZAK) then, but that did barely work for the Windows part - file transfers were slow, depended on the clients' hardware a lot (requiring fiddling with MS DOS network driver disks), and on the ZAK server the files for installing happened do disappear for no good reason every now and then. Not working well, and leaving out NetBSD (and everything elase), I created g4u. This gave me the (relative) pain of getting things working once, but with the option to easily add network drivers as they appeared in NetBSD (and oh they did!), plus allowed me to install any operating system.

The present: We've used g4u successfully in our labs then, booting from CDROM. I also got many donations from public and private instituations plus comanies from many sectors, indicating that g4u does make a difference.

In the mean time, the world has changed, and CDROMs aren't used that much any more. Network boot and USB sticks are today's devices of choice, cloning of a full disk without knowing its structure has both advantages but also disadvantages, and g4u's user interface is still command-line based with not much space for automation. For storage, FTP servers are nice and fast, but alternatives like SSH/SFTP, NFS, iSCSI and SMB for remote storage plus local storage (back to fun with filesystems, anyone? avoiding this was why g4u was created in the first place!) should be considered these days. Further aspects include integrity (checksums), confidentiality (encryption). This leaves a number of open points to address either by future releases, or by other products.

The future: At this point, my time budget for g4u is very limited. I welcome people to contribute to g4u - g4u is Open Source for a reason. Feel free to get back to me for any changes that you want to contribute!

The changes: Major changes in g4u 2.6 include:

The software: Please see the g4u homepage's download section on how to get and use g4u.

Enjoy!


February 09, 2017

BSD Talk bsdtalk266 - The nodes take over
We became tired of waiting. File Info: 7Min, 3MB. Ogg Link: https://archive.org/download/bsdtalk266/bsdtalk266.ogg

July 08, 2016

Frederic Cambus NetBSD on the CubieBoard2

Here are some notes on installing and running NetBSD/evbarm on the AllWinner A20 powered CubieBoard2. I bought this board a few weeks ago for its SATA capabilities, despite the fact that there are now cheaper boards with more powerful CPUs.

Required steps for creating a bootable micro SD card are detailed on the NetBSD Wiki, and a NetBSD installation is required to run mkubootimage.

I used an USB to TTL serial cable to connect to the board and create user accounts. Do not be afraid of serial, as it has in fact only advantages: there is no need to connect an USB keyboard nor an HDMI display, and it also brings back nice memories.

Connecting using cu (from my OpenBSD machine):

cu -s 115200 -l /dev/cuaU0

Device name might be different when using cu on other operating systems.

Adding a regular user in the wheel group:

useradd -m -G wheel username

Adding a password to the newly created user and changing default shell to ksh:

passwd username
chpass -s /bin/ksh username

Installing and configuring pkgin:

export PKG_PATH=http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/earmv7hf/7.0/All
pkg_add pkgin
echo $PKG_PATH > /usr/pkg/etc/pkgin/repositories.conf
pkgin update

Finally, here is a dmesg for reference purposes:

Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 7.0.1 (CUBIEBOARD.201605221355Z)
total memory = 1024 MB
avail memory = 1008 MB
sysctl_createv: sysctl_create(machine_arch) returned 17
kern.module.path=/stand/evbarm/7.0/modules
timecounter: Timecounters tick every 10.000 msec
mainbus0 (root)
cpu0 at mainbus0 core 0: 912 MHz Cortex-A7 r0p4 (Cortex V7A core)
cpu0: DC enabled IC enabled WB disabled EABT branch prediction enabled
cpu0: isar: [0]=0x2101110 [1]=0x13112111 [2]=0x21232041 [3]=0x11112131, [4]=0x10011142, [5]=0
cpu0: mmfr: [0]=0x10101105 [1]=0x40000000 [2]=0x1240000 [3]=0x2102211
cpu0: pfr: [0]=0x1131 [1]=0x11011
cpu0: 32KB/32B 2-way L1 VIPT Instruction cache
cpu0: 32KB/64B 4-way write-back-locking-C L1 PIPT Data cache
cpu0: 256KB/64B 8-way write-through L2 PIPT Unified cache
vfp0 at cpu0: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
vfp0: mvfr: [0]=0x10110222 [1]=0x11111111
cpu1 at mainbus0 core 1
armperiph0 at mainbus0
armgic0 at armperiph0: Generic Interrupt Controller, 160 sources (151 valid)
armgic0: 32 Priorities, 128 SPIs, 7 PPIs, 16 SGIs
armgtmr0 at armperiph0: ARMv7 Generic 64-bit Timer (24000 kHz)
armgtmr0: interrupting on irq 27
timecounter: Timecounter "armgtmr0" frequency 24000000 Hz quality 500
awinio0 at mainbus0: A20 (0x1651)
awingpio0 at awinio0
awindma0 at awinio0: DMA
awindma0: interrupting on irq 59
awincnt0 at awinio0
timecounter: Timecounter "CNT64" frequency 24000000 Hz quality 900
com0 at awinio0 port 0: ns16550a, working fifo
com0: console
awindebe0 at awinio0 port 0: Display Engine Backend (BE0)
awintcon0 at awinio0 port 0: LCD/TV timing controller (TCON0)
awinhdmi0 at awinio0: HDMI 1.3
awinwdt0 at awinio0: default period is 10 seconds
awinrtc0 at awinio0: RTC
awinusb0 at awinio0 port 0
awinusb0: no restrict gpio found
ohci0 at awinusb0: OHCI USB controller
ohci0: OHCI version 1.0
usb0 at ohci0: USB revision 1.0
ohci0: interrupting on irq 96
ehci0 at awinusb0: EHCI USB controller
ehci0: EHCI version 1.0
ehci0: companion controller, 1 port each: ohci0
usb1 at ehci0: USB revision 2.0
ehci0: interrupting on irq 71
awinusb1 at awinio0 port 1
awinusb1: no restrict gpio found
ohci1 at awinusb1: OHCI USB controller
ohci1: OHCI version 1.0
usb2 at ohci1: USB revision 1.0
ohci1: interrupting on irq 97
ehci1 at awinusb1: EHCI USB controller
ehci1: EHCI version 1.0
ehci1: companion controller, 1 port each: ohci1
usb3 at ehci1: USB revision 2.0
ehci1: interrupting on irq 72
motg0 at awinio0: OTG
motg0: interrupting at irq 70
motg0: no restrict gpio found
motg0: Dynamic FIFO sizing detected, assuming 16Kbytes of FIFO RAM
usb4 at motg0: USB revision 2.0
awinmmc0 at awinio0 port 0: SD3.0 (DMA)
awinmmc0: interrupting at irq 64
ahcisata0 at awinio0: AHCI SATA controller
ahcisata0: interrupting on irq 88
ahcisata0: ignoring broken port multiplier support
ahcisata0: AHCI revision 1.10, 1 port, 32 slots, CAP 0x6f24ff80<CCCS,PSC,SSC,PMD,SAM,ISS=0x2=Gen2,SCLO,SAL,SALP,SSS,SSNTF,SNCQ>
atabus0 at ahcisata0 channel 0
awiniic0 at awinio0 port 0: Marvell TWSI controller
awiniic0: interrupting on irq 39
iic0 at awiniic0: I2C bus
awge0 at awinio0: Gigabit Ethernet Controller
awge0: interrupting on irq 117
awge0: Ethernet address: 02:0a:09:03:27:08
rlphy0 at awge0 phy 1: RTL8201L 10/100 media interface, rev. 1
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
awinac0 at awinio0: CODEC
audio0 at awinac0: full duplex, playback, capture, mmap, independent
awinhdmiaudio0 at awinio0: HDMI 1.3
audio1 at awinhdmiaudio0: half duplex, playback, mmap
awinnand0 at awinio0
awinir0 at awinio0 port 0: IR
awinir0: interrupting on irq 37
cir0 at awinir0
gpio0 at awingpio0: 18 pins
gpio1 at awingpio0: 25 pins
gpio2 at awingpio0: 28 pins
gpio3 at awingpio0: 12 pins
gpio4 at awingpio0: 12 pins
gpio5 at awingpio0: 28 pins
gpio6 at awingpio0: 22 pins
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
cpu1: 912 MHz Cortex-A7 r0p4 (Cortex V7A core)
cpu1: DC enabled IC enabled WB disabled EABT branch prediction enabled
cpu1: isar: [0]=0x2101110 [1]=0x13112111 [2]=0x21232041 [3]=0x11112131, [4]=0x10011142, [5]=0
cpu1: mmfr: [0]=0x10101105 [1]=0x40000000 [2]=0x1240000 [3]=0x2102211
cpu1: pfr: [0]=0x1131 [1]=0x11011
cpu1: 32KB/32B 2-way L1 VIPT Instruction cache
cpu1: 32KB/64B 4-way write-back-locking-C L1 PIPT Data cache
cpu1: 256KB/64B 8-way write-through L2 PIPT Unified cache
vfp1 at cpu1: NEON MPE (VFP 3.0+), rounding, NaN propagation, denormals
vfp1: mvfr: [0]=0x10110222 [1]=0x11111111
sdmmc0 at awinmmc0
uhub0 at usb0: Allwinner OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1 at usb1: Allwinner EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub1: 1 port with 1 removable, self powered
uhub2 at usb2: Allwinner OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 1 port with 1 removable, self powered
uhub3 at usb3: Allwinner EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub3: 1 port with 1 removable, self powered
uhub4 at usb4: Mentor Graphics MOTG root hub, class 9/0, rev 1.00/1.00, addr 1
uhub4: 1 port with 1 removable, self powered
ld0 at sdmmc0: <0x02:0x544d:SA08G:0x14:0x12436e27:0x0e6>
ld0: 7388 MB, 3752 cyl, 64 head, 63 sec, 512 bytes/sect x 15130624 sectors
ld0: 4-bit width, bus clock 50.000 MHz
boot device: ld0
root on ld0a dumps on ld0b
root file system type: ffs