NetBSD Planet


August 08, 2020

DragonFly BSD Digest In Other BSDs for 2020/08/08

Fun variety this week.


August 07, 2020

Ruben Schade Trademarks disputes in tech

Trademark disputes and confusion have exited since they have; a tautological statement for which I couldn’t resist. But in the spirit of realising when I’ve been previously wrong about something, I’m here to explore the idea that there may be something to trademark antsiness in tech after all.

The term podcast was my first experience with this. Everyone I knew and respected in the tech and online media communities couldn’t stand the phrase when it came out in the mid 2000s. Frank Nora already had his New Time Radio moniker to describe downloadable online broadcasts. Jim Kloss famously used Internet Magazine at Whole Wheat Radio because he didn’t think the future only belonged to Apple. A former TechTV titan preferred netcast because it attributed the Internet as the enabling tech, like satellite radio.

(Incidently, this is the etymology behind the NetBSD name as well. I always thought it looked elegant compared to the other BSDs).

I used to think New Time Radio and Audio Magazine were more fun, clever, and relatable to a wider audience than podcast, but I’ll admit, deep down, I thought the association with Apple’s then-ubiquitous iPod was overblown. Despite what Apple’s legal team may have thought at the time, they didn’t invent the word pod. And besides, everyone knew they could download and listen to Ze Frank or Digital Flotsam wherever they wanted.

But just as I did with a new bottle of Sriracha Tabasco in a pasta dish, I grossly under-estimated the term’s impact. Years before podcasting became as cool and ubiquitous as it is now, I still fielded questions for my own silly show asking why they needed an Apple device to listen to it. I ended up linking to the BBC’s podcasting FAQ page where they took great pains to say it worked on your desktop, iRiver, flip phone, or anything else that supported MP3s.

Meanwhile GitHub was taking off as the Internet’s preferred code repository and collaboration platform. I still remember a lecturer at uni assuring us we didn’t need to use GitHub to use Git for our major assignment. I thought he was joking, but even afterwards we had group members who were confused. “GitHub is Git, right?” they all asked, before we decided on Subversion.

Another example is rsync.net. I recently noticed it among a vendor list on a client proposal. I thought they’d just listed the website for the indispensable file sync tool, but it referred to a commercial backup service. I realised my mistake, but how many other stakeholders were caught out too? Large companies can defend their trademarks, but I worry when I see the name of a free or open source tool being used in one. They may have the blessing of the tool’s creator—in some cases they may not have a choice—but it can introduce confusion, and the potential to assume that one endorses the other.

CentOS being Red Hat Enterprise Linux in all but name is one of the worst-kept secrets in the industry, so it’s clear it’s possible to build your own reputation and pedigree without piggy-backing off another title. I suppose that’s harder, though.


This post originally appeared on Rubenerd.

NetBSD Blog GSoC 2020 Second Evaluation Report: Curses Library Automated Testing
This report was prepared by Naman Jain as a part of Google Summer of Code 2020

My GSoC project under NetBSD involves the development of test framework of curses library. This blog report is second in series of blog reports; you can have a look at the first report. This report would cover the progress made in second coding phase along with providing some insights into the libcurses.

Complex characters

A complex character is a set of associated character, which may include a spacing character and non-spacing characters associated with it. Typical effects of non-spacing character on associated complex character c include: modifying the appearance of c (like adding diacritical marks) or bridge c with the following character. The cchar_t data type represents a complex character and its rendition. In NetBSD, this data type has following structure:

struct cchar_t {
	attr_t attributes; /* character attributes */
	unsigned elements; /* number of wide char in vals*/
	wchar_t vals[CURSES_CCHAR_MAX]; /* wide chars including non-spacing */
};

vals array contains the spacing character and associated non-spacing characters. Note that NetBSD supports wchar_t (wide character) due to which multi-byte characters are supported. To use the complex characters one has to correctly set the locale settings. In this coding period, I wrote tests for routines involving complex characters.

Alternate character set

When you print "BSD", you would send the hex-codes 42, 53, 44 to the terminal. Capability of graphic capable printers was limited by 8-bit ASCII code. To solve this, additional character sets were introduced. We can switch between the modes using escape sequence. One such character set for Special Graphics is used by curses for line drawing. In a shell you can type

echo -e "\e(0j\e(b"

to get a lower-right corner glyph. This enables alternate character mode (\e(), prints a character(j) and disables alternate character mode (\e(b). One might wonder where this 'j' to 'Lower Right Corner glyph' comes from. You may see that mapping ("acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,) via

infocmp -1 $TERM | grep acsc

These characters are used in box_set(), border_set(), etc. functions which I tested in the second coding period.

Progress in the second coding phase

Improvements in the framework:

  1. Added support for testing of functions to be called before initscr()
  2. Updated the unsupported function definitions with some minor bug fixes.

Testing and bug reports

  1. Added tests for following families of functions:
    • Complex character routines.
    • Line/box drawing routines.
    • Pad routines.
    • Window and sub-window operations.
    • Curson manipulation routines
  2. Reported bugs (and possible fixes if I know):
    • lib/55454 wredrawln() in libcurses does not follow the sensible behaviour [fixed]
    • lib/55460 copy error in libcurses [fixed]
    • lib/55474 wattroff() unsets all attributes if passed STANDOUT as argument [standard is not clear, so decided to have as it is]
    • lib/55482 slk_restore() does not restore the slk screen
    • lib/55484 newwin() results into seg fault [fixed]
    • lib/55496 bkgrnd() doesn't work as expected
    • lib/55517 wresize() function incorrectly resizes the subwindows

I would like to thank my mentors Brett and Martin, as well as the NetBSD community for their support whenever I faced some issues.


August 06, 2020

Frederic Cambus NetBSD on the NanoPi NEO2

The NanoPi NEO2 from FriendlyARM has been serving me well since 2018, being my test machine for OpenBSD/arm64 related things.

As NetBSD/evbarm finally gained support for AArch64 in NetBSD 9.0, released back in February, I decided to give it a try on this device. The board only has 512MB of RAM, and this is where NetBSD really shines. Things have become a lot easier since [email protected] now provides bootable ARM images for a variety of devices, including the NanoPi NEO2.

On first boot, the system will resize the filesystem to automatically expand to the size of the SD card.

Growing ld0 MBR partition #1 (1052MB -> 60810MB)
Growing ld0 disklabel (1148MB -> 60906MB)
Resizing /
/dev/rld0a: grow cg |************************************                 |  69%

Once the system is up and running, we can add a regular user in the wheel group:

useradd -m -G wheel username

And add a password to the newly created user:

passwd username

From there we do not need the serial console anymore and can connect to the device using SSH.

NetBSD has binary packages available for this architecture, and installing and configuring pkgin can be done as follow:

export PKG_PATH=https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/aarch64/9.0/All/
pkg_add pkgin
echo $PKG_PATH > /usr/pkg/etc/pkgin/repositories.conf
pkgin update

The base system can be kept up to date using sysupgrade, which can be installed via pkgin:

pkgin in sysupgrade

The following variable need to be set in /usr/pkg/etc/sysupgrade.conf:

RELEASEDIR="https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/evbarm-aarch64"

Lastly, the device has two user controllable LEDs which can be toggled on and off using sysctl.

To switch both LEDs on:

sysctl -w hw.led.nanopi_green_pwr=1
sysctl -w hw.led.nanopi_blue_status=1

To switch off the power LED automatically at boot time:

echo "hw.led.nanopi_green_pwr=0" >> /etc/sysctl.conf

Here is a dmesg for reference purposes:

[     1.000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
[     1.000000]     2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
[     1.000000]     2018, 2019, 2020 The NetBSD Foundation, Inc.  All rights reserved.
[     1.000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[     1.000000]     The Regents of the University of California.  All rights reserved.

[     1.000000] NetBSD 9.0_STABLE (GENERIC64) #0: Wed Aug  5 15:20:21 UTC 2020
[     1.000000] 	[email protected]:/usr/src/sys/arch/evbarm/compile/GENERIC64
[     1.000000] total memory = 497 MB
[     1.000000] avail memory = 479 MB
[     1.000000] timecounter: Timecounters tick every 10.000 msec
[     1.000000] armfdt0 (root)
[     1.000000] simplebus0 at armfdt0: FriendlyARM NanoPi NEO 2
[     1.000000] simplebus1 at simplebus0
[     1.000000] simplebus2 at simplebus0
[     1.000000] cpus0 at simplebus0
[     1.000000] simplebus3 at simplebus0
[     1.000000] psci0 at simplebus0: PSCI 1.1
[     1.000000] cpu0 at cpus0: Cortex-A53 r0p4 (Cortex V8-A core)
[     1.000000] cpu0: package 0, core 0, smt 0
[     1.000000] cpu0: IC enabled, DC enabled, EL0/EL1 stack Alignment check enabled
[     1.000000] cpu0: Cache Writeback Granule 16B, Exclusives Reservation Granule 16B
[     1.000000] cpu0: Dcache line 64, Icache line 64
[     1.000000] cpu0: L1 32KB/64B 2-way read-allocate VIPT Instruction cache
[     1.000000] cpu0: L1 32KB/64B 4-way write-back read-allocate write-allocate PIPT Data cache
[     1.000000] cpu0: L2 512KB/64B 16-way write-back read-allocate write-allocate PIPT Unified cache
[     1.000000] cpu0: revID=0x180, PMCv3, 4k table, 64k table, 16bit ASID
[     1.000000] cpu0: auxID=0x11120, FP, CRC32, SHA1, SHA256, AES+PMULL, NEON, rounding, NaN propagation, denormals, 32x64bitRegs, Fused Multiply-Add
[     1.000000] cpu1 at cpus0: Cortex-A53 r0p4 (Cortex V8-A core)
[     1.000000] cpu1: package 0, core 1, smt 0
[     1.000000] cpu2 at cpus0: Cortex-A53 r0p4 (Cortex V8-A core)
[     1.000000] cpu2: package 0, core 2, smt 0
[     1.000000] cpu3 at cpus0: Cortex-A53 r0p4 (Cortex V8-A core)
[     1.000000] cpu3: package 0, core 3, smt 0
[     1.000000] gic0 at simplebus1: GIC
[     1.000000] armgic0 at gic0: Generic Interrupt Controller, 224 sources (215 valid)
[     1.000000] armgic0: 16 Priorities, 192 SPIs, 7 PPIs, 16 SGIs
[     1.000000] fclock0 at simplebus2: 24000000 Hz fixed clock (osc24M)
[     1.000000] sunxisramc0 at simplebus1: SRAM Controller
[     1.000000] fclock1 at simplebus2: 32768 Hz fixed clock (ext_osc32k)
[     1.000000] gtmr0 at simplebus0: Generic Timer
[     1.000000] gtmr0: interrupting on GIC irq 27
[     1.000000] armgtmr0 at gtmr0: Generic Timer (24000 kHz, virtual)
[     1.000000] timecounter: Timecounter "armgtmr0" frequency 24000000 Hz quality 500
[     1.000010] sun8ih3ccu0 at simplebus1: H3 CCU
[     1.000010] sun8ih3rccu0 at simplebus1: H3 PRCM CCU
[     1.000010] sunxide2ccu0 at simplebus1: DE2 CCU
[     1.000010] sunxigpio0 at simplebus1: PIO
[     1.000010] gpio0 at sunxigpio0: 94 pins
[     1.000010] sunxigpio0: interrupting on GIC irq 43
[     1.000010] sunxigpio1 at simplebus1: PIO
[     1.000010] gpio1 at sunxigpio1: 12 pins
[     1.000010] sunxigpio1: interrupting on GIC irq 77
[     1.000010] fregulator0 at simplebus0: vcc3v3
[     1.000010] fregulator1 at simplebus0: usb0-vbus
[     1.000010] fregulator2 at simplebus0: gmac-3v3
[     1.000010] sun6idma0 at simplebus1: DMA controller (12 channels)
[     1.000010] sun6idma0: interrupting on GIC irq 82
[     1.000010] com0 at simplebus1: ns16550a, working fifo
[     1.000010] com0: console
[     1.000010] com0: interrupting on GIC irq 32
[     1.000010] sunxiusbphy0 at simplebus1: USB PHY
[     1.000010] sunxihdmiphy0 at simplebus1: HDMI PHY
[     1.000010] sunximixer0 at simplebus1: Display Engine Mixer
[     1.000010] sunxilcdc0 at simplebus1: TCON1
[     1.000010] sunxilcdc0: interrupting on GIC irq 118
[     1.000010] sunxirtc0 at simplebus1: RTC
[     1.000010] emac0 at simplebus1: EMAC
[     1.000010] emac0: Ethernet address 02:01:f7:f9:2f:67
[     1.000010] emac0: interrupting on GIC irq 114
[     1.000010] rgephy0 at emac0 phy 7: RTL8211E 1000BASE-T media interface
[     1.000010] rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
[     1.000010] h3codec0 at simplebus1: H3 Audio Codec (analog part)
[     1.000010] sunximmc0 at simplebus1: SD/MMC controller
[     1.000010] sunximmc0: interrupting on GIC irq 92
[     1.000010] motg0 at simplebus1: 'otg' mode not supported
[     1.000010] ehci0 at simplebus1: EHCI
[     1.000010] ehci0: interrupting on GIC irq 104
[     1.000010] ehci0: EHCI version 1.0
[     1.000010] ehci0: 1 companion controller, 1 port
[     1.000010] usb0 at ehci0: USB revision 2.0
[     1.000010] ohci0 at simplebus1: OHCI
[     1.000010] ohci0: interrupting on GIC irq 105
[     1.000010] ohci0: OHCI version 1.0
[     1.000010] usb1 at ohci0: USB revision 1.0
[     1.000010] ehci1 at simplebus1: EHCI
[     1.000010] ehci1: interrupting on GIC irq 110
[     1.000010] ehci1: EHCI version 1.0
[     1.000010] ehci1: 1 companion controller, 1 port
[     1.000010] usb2 at ehci1: USB revision 2.0
[     1.000010] ohci1 at simplebus1: OHCI
[     1.000010] ohci1: interrupting on GIC irq 111
[     1.000010] ohci1: OHCI version 1.0
[     1.000010] usb3 at ohci1: USB revision 1.0
[     1.000010] sunxiwdt0 at simplebus1: Watchdog
[     1.000010] sunxiwdt0: default watchdog period is 16 seconds
[     1.000010] /soc/[email protected] at simplebus1 not configured
[     1.000010] gpioleds0 at simplebus0: nanopi:green:pwr nanopi:blue:status
[     1.000010] /soc/[email protected] at simplebus1 not configured
[     1.000010] /soc/[email protected] at simplebus1 not configured
[     1.000010] timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
[     1.000010] cpu2: IC enabled, DC enabled, EL0/EL1 stack Alignment check enabled
[     1.000010] cpu2: Cache Writeback Granule 16B, Exclusives Reservation Granule 16B
[     1.040229] cpu2: Dcache line 64, Icache line 64
[     1.040229] cpu2: L1 32KB/64B 2-way read-allocate VIPT Instruction cache
[     1.050220] cpu2: L1 32KB/64B 4-way write-back read-allocate write-allocate PIPT Data cache
[     1.060220] cpu2: L2 512KB/64B 16-way write-back read-allocate write-allocate PIPT Unified cache
[     1.070220] cpu2: revID=0x180, PMCv3, 4k table, 64k table, 16bit ASID
[     1.070220] cpu2: auxID=0x11120, FP, CRC32, SHA1, SHA256, AES+PMULL, NEON, rounding, NaN propagation, denormals, 32x64bitRegs, Fused Multiply-Add
[     1.090221] cpu1: IC enabled, DC enabled, EL0/EL1 stack Alignment check enabled
[     1.090221] cpu1: Cache Writeback Granule 16B, Exclusives Reservation Granule 16B
[     1.100222] cpu1: Dcache line 64, Icache line 64
[     1.110221] cpu1: L1 32KB/64B 2-way read-allocate VIPT Instruction cache
[     1.110221] cpu1: L1 32KB/64B 4-way write-back read-allocate write-allocate PIPT Data cache
[     1.120222] cpu1: L2 512KB/64B 16-way write-back read-allocate write-allocate PIPT Unified cache
[     1.130222] cpu1: revID=0x180, PMCv3, 4k table, 64k table, 16bit ASID
[     1.140223] cpu1: auxID=0x11120, FP, CRC32, SHA1, SHA256, AES+PMULL, NEON, rounding, NaN propagation, denormals, 32x64bitRegs, Fused Multiply-Add
[     1.150222] cpu3: IC enabled, DC enabled, EL0/EL1 stack Alignment check enabled
[     1.160223] cpu3: Cache Writeback Granule 16B, Exclusives Reservation Granule 16B
[     1.160223] cpu3: Dcache line 64, Icache line 64
[     1.170223] cpu3: L1 32KB/64B 2-way read-allocate VIPT Instruction cache
[     1.180223] cpu3: L1 32KB/64B 4-way write-back read-allocate write-allocate PIPT Data cache
[     1.180223] cpu3: L2 512KB/64B 16-way write-back read-allocate write-allocate PIPT Unified cache
[     1.190223] cpu3: revID=0x180, PMCv3, 4k table, 64k table, 16bit ASID
[     1.200224] cpu3: auxID=0x11120, FP, CRC32, SHA1, SHA256, AES+PMULL, NEON, rounding, NaN propagation, denormals, 32x64bitRegs, Fused Multiply-Add
[     1.210224] sdmmc0 at sunximmc0
[     1.240225] uhub0 at usb0: NetBSD (0000) EHCI root hub (0000), class 9/0, rev 2.00/1.00, addr 1
[     1.240225] uhub0: 1 port with 1 removable, self powered
[     1.240225] uhub1 at usb2: NetBSD (0000) EHCI root hub (0000), class 9/0, rev 2.00/1.00, addr 1
[     1.250226] uhub1: 1 port with 1 removable, self powered
[     1.250226] uhub2 at usb1: NetBSD (0000) OHCI root hub (0000), class 9/0, rev 1.00/1.00, addr 1
[     1.260226] uhub2: 1 port with 1 removable, self powered
[     1.260226] uhub3 at usb3: NetBSD (0000) OHCI root hub (0000), class 9/0, rev 1.00/1.00, addr 1
[     1.275641] uhub3: 1 port with 1 removable, self powered
[     1.275641] IPsec: Initialized Security Association Processing.
[     1.350228] sdmmc0: SD card status: 4-bit, C10, U1, A1
[     1.350228] ld0 at sdmmc0: <0x03:0x5344:SC64G:0x80:0x0cd9141d:0x122>
[     1.360690] ld0: 60906 MB, 7764 cyl, 255 head, 63 sec, 512 bytes/sect x 124735488 sectors
[     1.370228] ld0: 4-bit width, High-Speed/SDR25, 50.000 MHz
[     1.990242] boot device: ld0
[     1.990242] root on ld0a dumps on ld0b
[     2.000243] root file system type: ffs
[     2.010242] kern.module.path=/stand/evbarm/9.0/modules
UnitedBSD NetBSD on Pinebook Pro

Hey all! Long time lurker first time posting.
Want to start with a big thank you to everyone on UnitedBSD, really like this community.

Regarding NetBSD on the Pinebook Pro, I'm curious to know if anyone has any tips and tricks or just would like to share how they run NetBSD on the PBP. I recently got my hands on the PBP and I'm in love with this little machine - it's very close to reach daily driver status, only things missing for me is stable wifi and tearfree video playback. The bwfm driver has improved a lot in CURRENT and only hangs sometimes - unless I stream video, then it usually hangs within a minute.

Would really appreciate some insight on how you guys are running NetBSD on the PBP and what you have done to make it run better.

Any general discussion regarding NetBSD and Pinebook Pro are also welcomed.

Cheers,
Carl


August 05, 2020

NetBSD Package System (pkgsrc) on DaemonForums firefox audio on NetBSD 9
Is there anyone on the forum who's done this recently? I had a working setup back on NetBSD 7, but have been using other OSes since then. Today I managed to get NetBSD 9 to install and run on my little Atomic Pi (Intel, not rpi) - altho it was a trip with UEFI.

Firefox 77.0 + pulseaudio + alsa seems not to be working like it was on version 7 of NetBSD. The command "pacmd list-sinks" is not seeing any audio devices, although I have a C-Media USB audio stick plugged into the Atomic. Supposedly the driver for a USB audio stick is uaudio, but it doesn't show up in a modstat listing, and can't be found by modload. Do I need to compile this driver? - I would think it's always a default option.

Suggestions?

BTW: Youtube with FF77 seems fairly decent on the Atomic (speed-wise) using NetBSD9 (but lame w/o audio).
NetBSD Blog GSoC Reports: Fuzzing Rumpkernel Syscalls, Part 2
This report was prepared by Aditya Vardhan Padala as a part of Google Summer of Code 2020

I have been working on Fuzzing Rumpkernel Syscalls. This blogpost details the work I have done during my second coding period.

Reproducing crash found in ioctl()

Kamil has worked on reproducing the following crash

Thread 1 "" received signal SIGSEGV, Segmentation fault.
pipe_ioctl (fp=<optimized out>, cmd=<optimized out>, data=0x7f7fffccd700)
    at /usr/src/lib/librump/../../sys/rump/../kern/sys_pipe.c:1108
warning: Source file is more recent than executable.
1108                            *(int *)data = pipe->pipe_buffer.cnt;
(gdb) bt
#0  pipe_ioctl (fp=<optimized out>, cmd=<optimized out>, data=0x7f7fffccd700)
    at /usr/src/lib/librump/../../sys/rump/../kern/sys_pipe.c:1108
#1  0x000075b0de65083f in sys_ioctl (l=<optimized out>, uap=0x7f7fffccd820, retval=<optimized out>)
    at /usr/src/lib/librump/../../sys/rump/../kern/sys_generic.c:671
#2  0x000075b0de6b8957 in sy_call (rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, 
    sy=<optimized out>) at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:65
#3  sy_invoke (code=54, rval=0x7f7fffccd810, uap=0x7f7fffccd820, l=0x75b0de126500, sy=<optimized out>)
    at /usr/src/lib/librump/../../sys/rump/../sys/syscallvar.h:94
#4  rump_syscall ([email protected]=54, [email protected]=0x7f7fffccd820, [email protected]=24, 
    [email protected]=0x7f7fffccd810)
    at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/rump.c:769
#5  0x000075b0de6ad2ca in rump___sysimpl_ioctl (fd=<optimized out>, com=<optimized out>, 
    data=<optimized out>) at /usr/src/lib/librump/../../sys/rump/librump/rumpkern/rump_syscalls.c:979
#6  0x0000000000400bf7 in main (argc=1, argv=0x7f7fffccd8c8) at test.c:15

in the rump using a fuzzer that uses pip2, dup2 and ioctl syscalls and specific arguments that are known to cause a crash upon which my work develops.

https://github.com/adityavardhanpadala/rumpsyscallfuzz/blob/master/honggfuzz/ioctl/ioctl_fuzz2.c

Since rump is a multithreaded process. Crash occurs in any of those threads. By using a core dump we can quickly investigate the crash and fetch the backtrace from gdb for verification however this is not viable in the long run as you would be loading your working directory with lots of core dumps which consume a lot of space. So we need a better way to reproduce crashes.

Crash Reproducers

Getting crash reproducers working took quite a while. If we look at HF_ITER() function in honggfuzz, it is a simple wrapper for HonggfuzzFetchData() to fetch buffer of fixed size from the fuzzer.

void HonggfuzzFetchData(const uint8_t** buf_ptr, size_t* len_ptr) {
.
.
.
.
    *buf_ptr = inputFile; 
    *len_ptr = (size_t)rcvLen;
.
.
}

And if we observe the attribute we notice that inputFile is mmaped.

//libhfuzz/fetch.c:26
    if ((inputFile = mmap(NULL, _HF_INPUT_MAX_SIZE, PROT_READ, MAP_SHARED, _HF_INPUT_FD, 0)) ==
        MAP_FAILED) {
        PLOG_F("mmap(fd=%d, size=%zu) of the input file failed", _HF_INPUT_FD,
            (size_t)_HF_INPUT_MAX_SIZE);
    }

So in a similar approach HF_ITER() can be modified to read input from a file and be mmapped so that we can reuse the reproducers generated by honggfuzz.

Attempts have been made to use getchar(3) for fetching the buffer via STDIN but for some unknown reason it failed so we switched to mmap(2)

So we overload HF_ITER() function whenever we require to reproduce a crash. I chose the following approach to use the reproducers. So whenever we need to reproduce a crash we just define CRASH_REPR.


static
void Initialize(void)
{
#ifdef CRASH_REPR
    FILE *fp = fopen(argv[1], "r+");
    data = malloc(max_size);
    fread(data, max_size, 1, fp);
    fclose(fp);
#endif
    // Initialise the rumpkernel only once.
    if(rump_init() != 0)
        __builtin_trap();
}

#ifdef CRASH_REPR
void HF_ITER(uint8_t **buf, size_t *len) {
        *buf = (uint8_t *)data;
        *len = max_size;
        return;
}
#else
EXTERN void HF_ITER(uint8_t **buf, size_t *len);
#endif

This way we can easily reproduce crashes that we get and get the backtraces.

Generating C reproducers

Now the main goal is to create a c file which can reproduce the same crash occuring due to the reproducer. This is done by writing all the syscall executions to a file with arguments so they can directly be compiled and used.

#ifdef CRASH_REPR
        FILE *fp = fopen("/tmp/repro.c","a+");
        fprintf(fp, "rump_sys_ioctl(%" PRIu8 ", %" PRIu64 ");\n",get_u8(),get_ioctl_request());
        fclose(fp);
#else    
        rump_sys_ioctl(get_u8(), get_ioctl_request());
#endif

I followed the same above method for all the syscalls that are executed. So I get a proper order of syscalls executed in native c code that I can simply reuse.

Obstacles

The number of times each syscall is executed before getting to a crash is quite high. So trying to perform a write to a file or STDOUT will create a lot of overhead when the number of syscalls executed are quite high. This method is good enough but a bit of optimization will make it even better.

To-Do

Finally I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their guidance and Kamil Rytarowski for his constant support whenever I needed it.

NetBSD Blog GSoC Reports: Enhancing Syzkaller support for NetBSD, Part 2
This report was prepared by Ayushi Sharma as a part of Google Summer of Code 2020

As a part of Google summer code 2020, I have been working on Enhance the Syzkaller support for NetBSD. This post summarises the work done in the past month.

For work done in the first coding period, you can take a look at the previous post.

Automation for enhancement

With an aim of increasing the number of syscalls fuzzed, we have decided to automate the addition of descriptions for syscalls as well as ioctl device drivers in a customised way for NetBSD.

Design

All the ioctl commands for a device driver in NetBSD are stored inside the /src/sys/dev/&ltdriver_name>/ folder. The idea is to get information related to a particular ioctl command by extracting required information from the source code of drivers. To achieve the same, we have broken down our project into majorly three phases.

  1. Generating preprocessed files
  2. Extracting information required for generating descriptions
  3. Conversion to syzkaller’s grammar
Design

Generating Preprocessed files

For a given preprocessed file, c2xml tool outputs the preprocessed C code in xml format. Further, the intermediate xml format descriptions would help us to smoothly transform the c code to syzkaller specific descriptions, in the last stage of this tool. We have used Bear as an aid for fetching commands to preprocess files for a particular driver. Bear generates a file called compile_commands.json which stores the commands used for compiling a file in json format. We then run these commands with ‘-E’ gcc flag to fetch the preprocessed files.These preprocessed files then serve as an input to the c2xml program.

Extractor

Definition of ioctl calls defined in header files of device driver in NetBSD can be broken down to:

ioctl

When we see it from syzkaller’s perspective, there are basically three significant parts we need to extract for adding description to syzkaller.

Description of a particular ioctl command acc to syzkaller’s grammar:

ioctl$FOOIOCTL(fd &ltfd_driver>, cmd const[FOOIOCTL], pt ptr[DIR, &ltptr_type>])
ioctl description
ioctl description

These definitions can be grepped from a device’s header files. The type information or description for pointer can then be extracted from the output files generated by c2xml. If the third argument is a struct, the direction of the pointer is determined with the help of fun() macros.

To-Do

The extracted descriptions have to be converted into syzkaller-friendly grammer. We plan to add support for syscalls too , which would ease the addition of complex compat syscalls. This would help us to increase the syzkaller’s coverage significantly.

Stats

Along with this, We have continued to add support for few more syscalls these include: Syscalls related to sockets have also been added. This has increased syscall coverage percentage to 50.35.

Atlast, I would like to thank my mentors - Cryo, Siddharth Muralee and Santhosh along with Kamil for their guidance and support. I am thankful to NetBSD community too along with Google for providing me such an amazing opportunity.


August 04, 2020

NetBSD Blog The GNU GDB Debugger and NetBSD (Part 3)
The NetBSD team of developers maintains two copies of GDB:

The base-system version of GDB (GPLv3) still relies on a set of local patches. I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.

GDB changes

I've written an integration of GDB with fork(2) and vfork(2) events. Unfortunately, this support (present in a local copy of GDB in the base-system) had not been merged so far, because there is a generic kernel regression with the pg_jobc variable. This variable can be called a reference counter of the number of processes within a process group that has a parent with control over a terminal. The semantics of this variable are not very well defined and in the result the number can become negative. This unexpected state of pg_jobc resulted in spurious crashes during kernel fuzzing. As a result new kernel assertions checking for non-negative pg_jobc values were introduced in order to catch the anomalies quickly. GDB as a ptrace(2)-based application happened to reproduce negative pg_jobc values quickly and reliably and this stopped the further adoption of the fork(2) and vfork(2) patch in GDB, until the pg_jobc behavior is enhanced. I was planning to include support for posix_spawn(3) events as well, as they are implemented as a first-class operation through a syscall, however this is also blocked by the pg_jobc blocker.

A local patch for GDB is stored here for the time being.

I've enable multi-process mode in the NetBSD native target. This enabled proper support for multiple inferiors and ptrace(2) assisted management of the inferior processes and their threads.

    
    (gdb) info inferior
      Num  Description       Connection           Executable
    * 1    process 14952     1 (native)           /usr/bin/dig
      2    <null>            1 (native)
      3    process 25684     1 (native)           /bin/ls
      4    <null>            1 (native)           /bin/ls

Without this change, additional inferiors could be already added, but not properly controlled.

I've implemented the xfer_partial TARGET_OBJECT_SIGNAL_INFO support for NetBSD. NetBSD implements reading and overwriting siginfo_t received by the tracee. With TARGET_OBJECT_SIGNAL_INFO signal information can be examined and modified through the special variable $_siginfo. Currently NetBSD uses an identical siginfo type on all architectures, so there is no support for architecture-specific fields.

(gdb) b main
Breakpoint 1 at 0x71a0
(gdb) r
Starting program: /bin/ps 

Breakpoint 1, 0x00000000002071a0 in main ()
(gdb) p $_siginfo
$1 = {si_pad = {5, 0, 0, 0, 1, 0 , 1, 0 }, _info = {_signo = 5, 
    _code = 1, _errno = 0, _pad = 0, _reason = {_rt = {_pid = 0, _uid = 0, _value = {sival_int = 1, 
          sival_ptr = 0x1}}, _child = {_pid = 0, _uid = 0, _status = 1, _utime = 0, _stime = 0}, 
      _fault = {_addr = 0x0, _trap = 1, _trap2 = 0, _trap3 = 0}, _poll = {_band = 0, _fd = 1}, 
      _syscall = {_sysnum = 0, _retval = {0, 1}, _error = 0, _args = {0, 0, 0, 0, 0, 0, 0, 0}}, 
      _ptrace_state = {_pe_report_event = 0, _option = {_pe_other_pid = 0, _pe_lwp = 0}}}}}

NetBSD, contrary to Linux and other BSDs, supports a ptrace(2) operation to generate a core(5) file from a running process. This operation is used in the base-system gcore(1) program. The gcore functionality is also delivered by GDB, and I have prepared new code for GDB to wire PT_DUMPCORE into the GDB code for NetBSD, and thus support GDB's gcore functionality. This patch is still waiting in upstream review. A local copy of the patch is here.

(gdb) r
Starting program: /bin/ps 

Breakpoint 1, 0x00000000002071a0 in main ()
(gdb) gcore
Saved corefile core.4378
(gdb) !file core.4378
core.4378: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), NetBSD-style, from 'ps', pid=4378, uid=1000, gid=100, nlwps=1, lwp=4378 (signal 5/code 1)

Plan for the next milestone

Rewrite the gdbserver support and submit upstream.

Ruben Schade FSF’s Free Software Gang almost included FreeBSD

The footer of the GNU website includes this friendly graphic inviting you to meet the Free Software Gang:

Free Software Gang graphc

Look closely and you can make out Beastie, the BSD mascot and basis of the FreeBSD logo, peeking out behind the GNU. And on the left you can see Puffy, the unmistakeable logo of OpenBSD.

(I don’t see NetBSD or DragonFly BSD anywhere, but let’s pretend the latter is buzzing around an orange flag atop the Inkscape mountain. They can always ride on an illumos phoenix to get there).

We may have our disagreements abut licensing, but anything that fosters collaboration is a good thing. Linux benefits from some of FreeBSD’s userland and utilities, everyone uses OpenSSH from the OpenBSD project, and desktop BSD users like me run GPL’d window managers or desktop environments.

The Free Software Gang page lists the following under Operating Systems and Friends:

Notice anything missing? Xfce and LXQt aside, of course.

Refusing to mention or link to the BSDs, despite padding out their own graphic with their logos, strikes me as a tad disingenuous. The Free Software Foundation consider their licences free and compatible with the GPL.

For an FSF site so preoccupied with licencing and software freedom, they literally violated—if only in spirit—the only clause in the modern BSD/ISC licence, which requires attribution. I’ve come to expect that more and more; free software either means GPL, or GTFO. Which is a shame, for the reasons stated earlier in this post.


This post originally appeared on Rubenerd.


August 02, 2020

Ruben Schade Social network CFO says iOS 14 to hurt tracking

Salvador Rodriguez reported some good news in CNBC last Thursday:

Facebook Chief Financial Officer David Wehner said on Thursday that upcoming changes to Apple’s iOS 14 operating system could hurt the social network’s ability to target ads to users.

With the update to its mobile devices, Apple will ask users if they want to let app developers track their activity across other apps and websites. Apple has not said when iOS 14 will launch, but it’s expected to roll out this year.

I’ll be interested to see how Android will respond to this, if at all. It’s traditionally lagged behind the industry along with Chrome, for obvious reasons. It’s interesting, and perhaps a bit sad, that so many of my purchasing and IT decisions are made based on something being less bad than something else on a metric I care about.

FreeBSD and NetBSD are probably the only platforms I use now because I enjoy them.


This post originally appeared on Rubenerd.

NetBSD General on DaemonForums Several questions on updating NetBSD
.
i have several questions about updating NetBSD, so i'm writing them one by one.
i'm currently running NetBSD 9.0, the "Formal Release", the "major RELEASE".

what (i think) i want is this: from "Chapter 33. Updating an existing system from sources".
https://www.netbsd.org/docs/guide/en/chap-updating.html


Quote:

"A common mechanism for upgrading a NetBSD system to a newer version is by rebuilding the system from sources and installing the results."
.
Quote:

"In particular, if you are running a stable NetBSD release in a production environment, you are encouraged to perform this procedure regularly in order to incorporate any security fixes that have been applied to the branch since its release."
this is what i actually want.
i'm not talking about upgrading across major releases. i'm not talking about upgrading from NetBSD 8.0 to NetBSD 9.0. or: upgrading from NetBSD 9.0 to NetBSD 10.0. No.

--
(MY FIRST QUESTION IS,) what is/are the easiest, most simple list of terminal commands to update the operating system: (quote) "in order to incorporate any security fixes that have been applied". example: for Debian Linux it's: "apt update", "apt upgrade".

So, following the guide: https://www.netbsd.org/docs/guide/en/chap-updating.html

i ran: "sysbuild build".
the process took 6 hours. here's the output:

Code:

make release started at:  Mon Jul 20 19:44:24 UTC 2020
make release finished at: Tue Jul 21 01:13:52 UTC 2020
===> Successful make release
===> build.sh ended:      Tue Jul 21 01:13:52 UTC 2020
===> Summary of results:
        build.sh command:    ./build.sh -D/root/sysbuild/amd64/destdir -M/root/sysbuild/amd64/obj -N2 -R/root/sysbuild/release -T/root/sysbuild/amd64/tools -U -X/usr/xsrc -j2 -mamd64 -x release
        build.sh started:    Mon Jul 20 19:44:15 UTC 2020
        NetBSD version:      9.0_STABLE
        MACHINE:            amd64
        MACHINE_ARCH:        x86_64
        Build platform:      NetBSD 9.0 amd64
        HOST_SH:            /bin/sh
        No $TOOLDIR/bin/nbmake, needs building.
        Bootstrapping nbmake
        MAKECONF file:      /etc/mk.conf (File not found)
        TOOLDIR path:        /root/sysbuild/amd64/tools
        DESTDIR path:        /root/sysbuild/amd64/destdir
        RELEASEDIR path:    /root/sysbuild/release
        Created /root/sysbuild/amd64/tools/bin/nbmake
        Updated makewrapper: /root/sysbuild/amd64/tools/bin/nbmake-amd64
        Successful make release
        build.sh ended:      Tue Jul 21 01:13:52 UTC 2020
===> .
sysbuild: I: Command finished successfully

.
then, following the guide, i ran:
Code:

sysupgrade auto ~/sysbuild/release/$(uname -m)
Code:

sysupgrade auto
,
both of them failed. output:
.

Code:

sysupgrade: I: Starting auto-update with stages: fetch modules kernel sets etcupdate postinstall clean
sysupgrade: I: Linking local /root/sysbuild/release/amd64/binary/sets/base.tgz into /var/cache/sysupgrade
sysupgrade: E: Cannot open /root/sysbuild/release/amd64/binary/sets/base.tgz

--
(MY SECOND QUESTION IS,) what is causing these errors? and is just running "sysbuild build" is enough, and do i not need to run "sysupgrade" (in this case)? but the guide says so!

--
(MY THIRD QUESTION IS,) can someone please explain the difference between "sysbuild" and "sysupgrade" in 1-2 sentence (easy way)?

--
(MY FOURTH QUESTION IS,) is there not any kind of result report sheet, like:
Quote:

"you were running version/build "01" before, and now it has been updated to build/version "02". and these <a>, <b> and <c> security vulnerabilities has been fixed" ?

where do i get something like that?

--
(MY FIFTH QUESTION IS,) in Debian Linux, a simple "apt update", "apt upgrade" took couple of minutes only. is not there any faster method to achieve this in NetBSD? i have to run this 6 hours "sysbuild build" process again and again?

--
(MY SIXTH QUESTION IS,) it is written: (quote) "you are encouraged to perform this procedure regularly". well how much regularly? once in a month? twice in a month?

--
(MY SEVENTH QUESTION IS,) how do i know, by running a simple terminal command, that the NetBSD team has released a security update, and i better update soon? . or do i have to check out the following webpages for this purpose, say once weekly, and will that be enough?


https://mail-index.netbsd.org/security-announce/
https://mail-index.netbsd.org/netbsd-announce/
https://www.netbsd.org/support/security/advisory.html
https://www.netbsd.org/changes/

so just following these links on weekly basis is enough?
what i found out till now is, since the release of NetBSD 9.0, there's been only one single security announcement, titled: "Security Advisory 2020-002: Specific ICMPv6 error message packet can crash the system".


https://mail-index.netbsd.org/securi...msg000139.html
https://ftp.netbsd.org/pub/NetBSD/se...20-002.txt.asc

and it is clearly mentioned: "NetBSD 9.0: not affected".

also in the "NetBSD 9.0 Security Advisories" page:
https://www.netbsd.org/support/secur...tches-9.0.html

"the list of advisories applicable to the NetBSD 9.0 release: None yet."

--
(MY EIGHTH QUESTION IS,) so although it is instructed in the guide: (quote) "you are encouraged to perform this procedure regularly"; but if i'm running NetBSD 9.0 (the "Formal Release") (the "major RELEASE"), and although it's been 6 months since NetBSD 9.0 released, as there's been no vulnerability announcement, so i do not need to perform any sysbuild-sysupgrade, right?

i want to know the easiest, most simple method.
THANK YOU, FOR TAKING THE TIME TO READ THIS LONG QUESTIONNAIRE.
,

July 29, 2020

UnitedBSD Porter's wiki

Setting up NetBSD workstation for PKGSRC

Table of Contents

Prerequisites

UnitedBSD NetBSD Porter's wiki

Setting up NetBSD workstation for PKGSRC

Help edit wiki
Table of Contents

Prerequisites


July 28, 2020

NetBSD General on DaemonForums NetBSD router
I have NetBSD on a Raspberry Pi that I want to make into a router.

I have a USB wifi adapter plugged into it that is connected to the internet
And I have a USB to Ethernet adapter plugged in as well, with an Ethernet cable plugged into that adapter going to another computer (for now OpenBSD, but I want to be able to send the traffic not just directly to another computer but maybe to a switch or wireless access point).

(NetBSD) ifconfig urtwn0 (the wifi adapter) output
Code:

urtwn0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        ssid ComComs_5D74 nwkey *****
        powersave off
        bssid b0:be:76:58:5d:74 chan 2
        address: 74:da:38:f1:9e:16
        media: IEEE802.11 autoselect (OFDM54 mode 11g)
        status: active
        inet6 fe80::2be7:73a5:1b3e:5e2c%urtwn0/64 flags 0x0 scopeid 0x5
        inet 192.168.0.117/24 broadcast 192.168.0.255 flags 0x0

(NetBSD) ifconfig ure0 (USB to Ethernet adapter) output
Code:

ure0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
        capabilities=3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
        capabilities=3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
        enabled=0
        ec_capabilities=1<VLAN_MTU>
        ec_enabled=0
        address: 00:e0:4c:36:36:f0
        media: Ethernet autoselect (none)
        status: no carrier

(OpenBSD) ifconfig fxp0 (Ethernet) output
Code:

fxp0: flags=808843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,AUTOCONF4> mtu 1500
        lladdr 00:00:39:0a:c2:34
        index 1 priority 0 llprio 3
        media: Ethernet autoselect (none)
        status: no carrier


I have added
Code:

net.inet.ip.forwarding=1
to /etc/sysctl.conf
and
Code:

gateway_enable="YES"
to /etc/rc.conf
But I don't think it did anything.

How do I ``point'' ure0 traffic to and from urtwn0?
I don't need a complete guide on how to do this (but I'll take one if available), I would be happy to just be pointed in the right direction with what man pages I should read, or keywords to search.
Ruben Schade Finding file duplicates with fdupes on FreeBSD

Today I learned of the MIT-licenced fdupes by Adrian Lopez that performs hashed and byte-for-byte file comparisons and presents lists of duplicated files. It works well, and you should use it. A big shoutout to [email protected] for maintaining the FreeBSD port, and ef at Bonn University for maintaining on pkgsrc.

On my Mac I use the excellent open source dupeGuru GUI application, but I had a need to find duplicates across terabytes of data on one of my FreeBSD microservers over the weekend. I wanted a tool that I could easily run in a detached screen session, and fdupes fit the bill like a buff platypus. What?

I took a ZFS snapshot of my dataset in case things went pear-shaped and I needed to roll back, then set it to auto-delete duplicates in a my target directory. Substitute your pool, dataset, and directory as required:

# zfs snapshot pool/[email protected]$ echo "THIS WILL DELETE DATA"$ fdupes -r -d -N $directory

If you just want it to identify duplicates:

$ fdupes -r $directory > dupes.log

Or if you want it to prompt you as it finds them:

$ fdupes -r -d $directory

Someone will probably tell me that ZFS has deduping, but it’s not applicable in this case. This was just a quick and dirty job to clean up some recursively rsync’d mess that I did while half-asleep; and I just use lz4 compression for all my pools now anyway.

I could have cleaned it myself, but why not let The Machine™ do it?


This post originally appeared on Rubenerd.


July 27, 2020

UnitedBSD Daily packages for NetBSD/amd64 current pkgsrc

Hey everybody,

I needed a NetBSD-current system for testing pkgin changes, and
figured I may as well also set it up for daily package builds.

So if anybody would like a repository for the latest packages then
head over to https://pkgsrc.joyent.com/install-on-netbsd/ to install
the bootstrap kit.

All packages are signed, the mozilla-rootcerts certificates are
shipped to enable HTTPS, and I enabled a bunch of PKG_OPTIONS by
default. Feel free to request further additions.

Hope you find it useful.

Cheers,

--
Jonathan Perkin - Joyent, Inc. - www.joyent.com

Source: pkgsrc-users

Ruben Schade nvi2 in FreeBSD ports

Craig Leres on the FreeBSD ports team has deprecated nvi-devel in lieu of nvi2, with the former expiring this time in October. From the pkg-descr:

vi is an implementation of the ex/vi text editor. The original vi was written by William Joy. Later Mark Horton added a number of enhancements. nvi was written by Keith Bostic and was distributed as part of theFourth Berkeley Software Distribution (4BSD) by the University ofCalifornia, Berkeley. This version is based on a fork of nvi by Sven Verdoolaege.

Nie Alarie added nvi 2.1.3 to pkgsrc in June.

nvi is a distinct branch of vi, not to be confused with Vim or its offshoots. You can install Vim from ports—and I do—but nvi is the default editor family in the BSDs, so its useful to be sufficiently proficient in it. You may find learning nvi helps you be a better Vim user, too.

I taught myself nvi while I was at uni in 2010, which I’ve just realised is a decade ago. Aiyo.


This post originally appeared on Rubenerd.

NetBSD Package System (pkgsrc) on DaemonForums pkgsrc optimization flags ignored
Hello all,

I gain the impression my optimization flags are ignored. This is from my /etc/mk.conf

Code:

CPUFLAGS+=          -march=native DDDDDD1
COPTS+=              -g -O2 -pipe CCCCCC1

I would expect the compilation to fail due to the Ds and Cs. It does not. I also tried setting bogus CFLAGS.

A grep for these flags does not show up in the build directory, either.

Code:

# grep -R DDDDDD1 /var/tmp/pkgsrc/graphics/xli/work/
# grep -R CCCCCC1 /var/tmp/pkgsrc/graphics/xli/work/

I tried graphics/xli as pkgsrc's xli Makefile is quite simple making it unlikely to overwrite mk.conf variables.

July 25, 2020

UnitedBSD NetBSD datacenter outage there's an Amiga running a mirror

A couple of major NetBSD servers are down due to a datacenter outage.
If you miss the website, there's an amiga running a mirror at http://de.netbsd.org

https://mobile.twitter.com/netbsd/status/1286898183923277829

DragonFly BSD Digest In Other BSDs for 2020/07/25

Lots of old BSD this week.


July 23, 2020

NetBSD General on DaemonForums pxe boot: I'm confused :((
Hello!

According to the man pxeboot, you need to declare in the host section
Code:

    host myhost {
        hardware ethernet 00:00:00:00:00:00;
        fixed-address myhost;
        option host-name "myhost";
        next-server mytftpserver;

        # This section allows dhcpd to respond with different answers
        # for the different tftp requests for the bootloader and kernel.
        if substring (option vendor-class-identifier, 0, 20)
          = "PXEClient:Arch:00000" {
            filename "pxeboot_ia32.bin";
        } elsif substring (option vendor-class-identifier, 0, 17)
          = "NetBSD:i386:libsa" {
            if filename = "boot.cfg" {
                filename "tftp:boot.cfg";
            } else if filename = "netbsd" {
                filename "tftp:netbsd-INSTALL.gz";
            }
        }
    }


But what if I want to load any host from the range (with a previously unknown client mac)?
But even if mac address is known, it is acceptable for one, two.. ten hosts.. What if there are 100, 200, 1000 of them?




If place an declare pxeboot options above the host directive

Code:

subnet 10.0.0.0 netmask 255.255.255.0 {
  interface tap1;
  range 10.0.0.200 10.0.0.230;
  option broadcast-address 10.0.0.255;
  option routers 10.0.0.1;
  option time-servers 10.0.0.1;
  option domain-name "home.local";
  option domain-search "home.local";
  ddns-domainname "home.local";

        # This section allows dhcpd to respond with different answers
        if substring (option vendor-class-identifier, 0, 20)
          = "PXEClient:Arch:00000" {
            filename "pxeboot_ia32.bin";
        } elsif substring (option vendor-class-identifier, 0, 17)
          = "NetBSD:i386:libsa" {
            if filename = "boot.cfg" {
                filename "tftp:boot.cfg";
            } else if filename = "netbsd" {
                filename "tftp:netbsd-INSTALL.gz";
            }
        }

  host tst1 {
          hardware ethernet 10:20:30:40:50:61;
#          hardware ethernet 10:20:30:40:50:60;
          fixed-address 10.0.0.60;
  }
}

then we get an error: "bootp: no reply".... :(

Attached Images
File Type: jpg bad.jpg (27.0 KB)

July 18, 2020

DragonFly BSD Digest In Other BSDs for 2020/07/18

There’s a separate Summer of Code section this week.


July 16, 2020

NetBSD Blog GSoC Reports: Benchmarking NetBSD, first evaluation report

This report was written by Apurva Nandan as part of Google Summer of Code 2020.

My GSoC project under NetBSD involves developing an automated regression and performance test framework for NetBSD that offers reproducible benchmarking results with detailed history and logs across various hardware & architectures.

To achieve this performance testing framework, I am using the Phoronix Test Suite (PTS) which is an open-source, cross-platform automated testing/benchmarking software for Linux, Windows and BSD environments. It allows the creation of new tests using simple XML files and shell scripts and integrates with revision control systems for per-commit regression testing.

About PTS

PTS core

PTS core is the engine of the Phoronix Test Suite and handles the following jobs:

Test-profiles/Suites & OpenBenchmarking

Test-profiles are light-weight XMLs and shell scripts that allow easy installation of the benchmarking packages and their evaluation. Test-profiles generally contains the following files:

A simple test-profile C-Ray-1.2.0 can be seen to get an idea of the XMLs and shell scripts.

Test-suites are bundles of related test-profiles or more suites, focusing on a subsystem or certain category of tests e.g. Disk Test Suite, Kernel, CPU suite, etc.

OpenBenchmarking.org serves as a repository for storing test profiles, test suites, hence allowing new/updated tests to be seamlessly obtained via PTS. OpenBenchmarking.org also provides a platform to store the test result data openly and do a comparison between any test results stored in the OpenBenchmarking.org cloud.

Usage

Test installation

The command:

$ phoronix-test-suite install test-profile-name

Fetches the sources of the tests related to the the test-profile in ~/.phoronix-test-suite/installed-tests, applies patches and carries out compilation of test sources for generating the executables to run the tests.

Test execution

The command:

$ phoronix-test-suite run test-profile-name

Performs installation of the test (similar to $ phoronix-test-suite install test-profile-name) followed by exectution the binary from the compiled sources of the test in ~/.phoronix-test-suite/installed-tests and finally reports the tests results/outcome to the user and provides an option to upload results to OpenBenchmarking.org.

Progress in the first phase of GSoC

pkgsrc-wip

The first task performed by me was upgrading the Phoronix Test Suite from version 8.8 to the latest stable version 9.6.1 in pkgsrc-wip and is available as wip/phoronix-test-suite. You can have a look at the PTS Changelog to know about the improvements between these two versions.

Major Commits:

Currently, the PTS upgrade to 9.6.1 is subject to more modifications and fixes before it gets finally merged to the pkgsrc upstream. To test the Phoronix Test Suite 9.6.1 on NetBSD till then, you can setup pkgsrc-wip on your system via:

$ cd /usr/pkgsrc
$ git clone git://wip.pkgsrc.org/pkgsrc-wip.git wip

To learn more about pkgsrc-wip please see The pkgsrc-wip project homepage.

After setting up pkgsrc-wip, use the following command for installation of PTS 9.6.1:

$ cd /usr/pkgsrc/wip/phoronix-test-suite
$ make install

If any new build/installation errors are encountered, please document them in wip/phoronix-test-suite/TODO file and/or contact me.

Testing & Debugging

As we now know, having a look over OpenBenchmarking.org’s available test-profiles section or using $ phoronix-test-suite list-available-tests, there are 309 test-profiles available on PTS for Linux, and only 166 of 309 tests have been ported to NetBSD (can be differentiated by a thunder symbol on the test profile on the website). These 166 test-profiles can be fetch, build and executed on NetBSD using just a single command $ phoronix-test-suite run test-profile-name. I am ignoring the graphics ones in both Linux & NetBSD as GPU benchmarking wasn’t required in the project.

Many of the test profiles available on NetBSD have installation, build, or runtime errors i.e. they don’t work out of the box using the command $ phoronix-test-suite run test-profile-name. I ran 90 of these test profiles on a NetBSD-current x86_64 QEMU VM (took a lot of time due to the heavy tests) and have reported the status in the sheet: NetBSD GSoC: Test Porting Progress Report

You can have a look at how a PTS test-profile under action looks like!

Aircrack-NG test-profile demonstration

Aircrack-ng is a tool for assessing WiFi/WLAN network security.

The PTS test-profile is available at: https://openbenchmarking.org/test/pts/aircrack-ng

Screenshot of aircrack-ng PTS test-profile results

For further information, complete results can be seen at https://openbenchmarking.org/result/2007116-NI-AIRCRACKN51

AOBench test-profile demonstration

AOBench is a lightweight ambient occlusion renderer, written in C.

The PTS test-profile is available at: https://openbenchmarking.org/test/pts/aircrack-ng

Screenshot of aobench PTS test-profile results

For further information, complete results can be seen at: https://openbenchmarking.org/result/2007148-NI-AOBENCHTE94

Debugging and fixing non-working test-profiles

After testing these test-profiles, I debugged the following build errors in the following test-profiles:

Fixes to the above bugs can be found in the sheet or in the GitHub repositories shared in the next paragraph.

The modified test-profiles have been pushed to pts-test-profiles-dev and the fix patches to pts-test-profiles-patches. These patches are automatically downloaded by the debugged test-profiles and automatically applied before building of tests. The steps to install the debugged test-profiles have been added in the README.md of pts-test-profiles-dev.

These patches have been added in the downloads.xml of the modified test-profiles, and hence they get fetched during the test installation. The install.sh script in these modified test-profiles applies them on the benchmarking packages if the operating system is detected as NetBSD, thereby removing the build errors.

coremark test-profile demonstration

You can have a look at the patched coremark-1.0.0 test-profile under execution:

The patched PTS test-profile is available at: https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/coremark-1.0.0

This is a test of EEMBC CoreMark processor benchmark.

Screenshot of patched coremark PTS test-profile results

For further information, complete results can be seen at: https://openbenchmarking.org/result/2007148-NI-LOCALCORE64

Porting more test-profiles to NetBSD

Attempts were made to port new test-profiles from Linux to NetBSD, but the major incompatibility issue was missing external dependencies in NetBSD. Hence, this may reduce the number of test-profiles we can actually port, but more sincere efforts will be made in this direction in the next phase.

Future Plans

My next steps would involve porting the non-available tests to NetBSD i.e. the non-graphics ones available on Linux but unavailable on NetBSD, and fix the remaining test-profiles for NetBSD.

After gaining an availability of a decent number of test-profiles on NetBSD, I will use Phoromatic, a remote tests management system for the PTS (allows the automatic scheduling of tests, remote installation of new tests, and the management of multiple test systems) to host the live results of the automated benchmarking framework on benchmark.NetBSD.org, thereby delivering a functional performance and regression testing framework.


July 11, 2020

Nikita Gillmann GSoC 2020 - Notes on sh(1)

Misc notes on the 3 relevant files for adding posix_spawn usage to sh, from https://github.com/teknokatze/gsoc2020/blob/default/sh.txt and summary from today's IRC meeting with my mentors. This week and last week I've read through sh's code, with no clue how to start replacing the logic.

We have 3 files (jobs, exec, eval) which seem to contain the main logic i need to investigate. Between fork and exec we sometimes fork before we exec and have too much disconnection between those two, or sometimes we exec in a vforked shell, and I need to understand more about what is going on or just start testing.

According to Riastradh the shell is going to seem a bit more complicated because it often forks subshells without directly reducing to exec. The low-hanging fruit is where it vforks, because you can't do much besides exec after vfork -- so that's the path that is already an opportunity for posix_spawn, because pretty much the only things it will do are set up signals and file descriptors and exec. In evalcommand, the relevant path is limited to the case where cmdentry.cmdtype == CMDNORMAL and (!cmd->ncmd.backgnd or cmd->ncmd.redirect == NULL), in which case it does listmklocal, forkchild, redirect, and shellexec. It's really down to what's between VFORK_BLOCK, and the case CMDNORMAL of switch (cmdentry.cmdtype) down below (which is the 'default' of the switch). This is mostly just redirect and shellexec. The easiest approach would probably be to rip out everything in VFORK_BLOCK/END and replace it by posix_spawn, don't bother falling through to the switch below; just incorporate the redirect and shellexec into posix_spawn there.


July 04, 2020

DragonFly BSD Digest In Other BSDs for 2020/07/04

This should be prime convention season, darnit.

Also on:

Twitter Twitter


June 30, 2020

Nikita Gillmann GSoC 2020 Report 1

This is my first report for the Google Summer of Code project I am working on for NetBSD.

Update 2020-07-14: The more polished version has been published and reviewed at our TNF blog.

Update 2020-07-10: Prior work. In GSoC 2012 Charles Zhang added the posix_spawn syscall which according to its SF repository at the time (maybe even now, I have not looked very much into comparing all other systems and libcs + kernels) is an in-kernel implementation of posix_spawn which provides performance benefits compared to FreeBSD and other systems which had a userspace implementation.

The code snippets in this article are under the same license as their respective files, ie 3-clause BSD licensed, Copyright (c) 1988, 1993 The Regents of the University of California. For full details refer to netbsd.org/about/redistribution and the full source code of the functions described. This is not legal advice and I'm only the author of (some portions of) the new version of the functions.

After 1 week of reading POSIX and writing code, 2 weeks of coding and another 1.5 weeks of bugfixes I have succesfully implemented posix_spawn in usage in system(3) and popen(3) internally.

The biggest challenge for me was to understand POSIX, to read the standard. I am used to reading more formal books, but I can't remember working with the posix standard directly before.

The next part of my Google Summer of Code project will focus on similar rewrites of NetBSD's sh(1).

system(3)

The prototype

int system(const char *command);

remains the same. Below I'm just commenting on the differences, not the whole function, and therefore only include code block where the versions differ the most. The full work can be found at gsoc2020 as well as src and will be submitted for inclusion later in the project.

Previously we'd use vfork, sigaction, and execve in this stdlib function.

The biggest difference to the 2015 version of our system version is the usage of posix_spawnattr_ where we'd use sigaction before, and posix_spawn where execve executes the command in a vfork'd child:

   posix_spawnattr_init(&attr);
   posix_spawnattr_setsigmask(&attr, &omask);
   posix_spawnattr_setflags(&attr, POSIX_SPAWN_SETSIGDEF|POSIX_SPAWN_SETSIGMASK);
   (void)__readlockenv();
   status = posix_spawn(&pid, _PATH_BSHELL, NULL, &attr, __UNCONST(argp), environ);
   (void)__unlockenv();
   posix_spawnattr_destroy(&attr);

The full version can be found here.

The prototype of posix_spawn is:

int posix_spawn(pid_t *restrict pid, const char *restrict path, const posix_spawn_file_actions_t *file_actions, const posix_spawnattr_t *restrict attrp, char *const argv[restrict], char *const envp[restrict]);

We first initialize a spawn attributes object with the default value for all of its attributes set. A spawn attributes object is used to modify the behavior of posix_spawn().

The previous fork-exec switch included calls to sigaction to set the behavior associated with SIGINT and SIGQUIT as defined by POSIX:

The system() function shall ignore the SIGINT and SIGQUIT signals, and shall block the SIGCHLD signal, while waiting for the command to terminate. If this might cause the application to miss a signal that would have killed it, then the application should examine the return value from system() and take whatever action is appropriate to the application if the command terminated due to receipt of a signal. source: https://pubs.opengroup.org/onlinepubs/9699919799/functions/system.html

This has been achieved with a combination of posix_spawnattr_setsigmask() and posix_spawnattr_setflags() in the initialized attributes object referenced by attr.

As before we call __readlockenv() and then call posix_spawn() which returns the process ID of the child process in the variable pointed to by 'pid', and returns zero as the function return value.

The old code:

   (void)__readlockenv();
   switch(pid = vfork()) {
   case -1:                        /* error */
           (void)__unlockenv();
           sigaction(SIGINT, &intsa, NULL);
           sigaction(SIGQUIT, &quitsa, NULL);
           (void)sigprocmask(SIG_SETMASK, &omask, NULL);
           return -1;
   case 0:                         /* child */
           sigaction(SIGINT, &intsa, NULL);
           sigaction(SIGQUIT, &quitsa, NULL);
           (void)sigprocmask(SIG_SETMASK, &omask, NULL);
           execve(_PATH_BSHELL, __UNCONST(argp), environ);
           _exit(127);
   }
   (void)__unlockenv();

popen(3), popenve(3)

As with system, the prototype of both functions remains the same:

FILE * popenve(const char *cmd, char *const *argv, char *const *envp, const char *type); FILE * popen(const char *cmd, const char *type);

Since the popen and popenve implementation in NetBSD's libc use a couple of shared helper functions, I was able to change both functions while keeping the majority of the changes focused on (some of ) the helper functions (pdes_child).

pdes_child, an internal function in popen.c, now takes one more argument (const char *cmd) for the command to pass to posix_spawn which is called in pdes_child.

pdes_child previously looked like this:

static void
pdes_child(int *pdes, const char *type)
{
        struct pid *old;

        /* POSIX.2 B.3.2.2 "popen() shall ensure that any streams
           from previous popen() calls that remain open in the 
           parent process are closed in the new child process. */
        for (old = pidlist; old; old = old->next)
#ifdef _REENTRANT
                (void)close(old->fd); /* don't allow a flush */
#else
                (void)close(fileno(old->fp)); /* don't allow a flush */
#endif

        if (type[0] == 'r') {
                (void)close(pdes[0]);
                if (pdes[1] != STDOUT_FILENO) {
                        (void)dup2(pdes[1], STDOUT_FILENO);
                        (void)close(pdes[1]);
                }
                if (type[1] == '+')
                        (void)dup2(STDOUT_FILENO, STDIN_FILENO);
        } else {
                (void)close(pdes[1]);
                if (pdes[0] != STDIN_FILENO) {
                        (void)dup2(pdes[0], STDIN_FILENO);
                        (void)close(pdes[0]);
                }
        }
}

My current blog layout and formating will also mess up the readability of this function, this is the new version (the whole file is here):

static int
pdes_child(int *pdes, const char *type, const char *cmd)
{
        struct pid *old;
        posix_spawn_file_actions_t file_action_obj;
        pid_t pid;
        const char *argp[] = {"sh", "-c", NULL, NULL};
        argp[2] = cmd;
        int error;

        error = posix_spawn_file_actions_init(&file_action_obj);
        if (error) {
                goto fail;
        }
        /* POSIX.2 B.3.2.2 "popen() shall ensure that any streams
           from previous popen() calls that remain open in the
           parent process are closed in the new child process. */
        for (old = pidlist; old; old = old->next)
#ifdef _REENTRANT
        error = posix_spawn_file_actions_addclose(&file_action_obj, old->fd); /* don't allow a flush */
        if (error) {
                goto fail;
        }
#else
        error = posix_spawn_file_actions_addclose(&file_action_obj, fileno(old->fp)); /* don't allow a flush */
        if (error) {
                goto fail;
        }
#endif
        if (type[0] == 'r') {
                error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[0]);
                if (error) {
                        goto fail;
                }
                if (pdes[1] != STDOUT_FILENO) {
                        error = posix_spawn_file_actions_adddup2(&file_action_obj, pdes[1], STDOUT_FILENO);
                        if (error) {
                                goto fail;
                        }
                        error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[1]);
                        if (error) {
                                goto fail;
                        }
                }
                if (type[1] == '+') {
                        error = posix_spawn_file_actions_adddup2(&file_action_obj, STDOUT_FILENO, STDIN_FILENO);
                        if (error) {
                                goto fail;
                        }
                }
        } else {
                error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[1]);
                if (error) {
                        goto fail;
                }
                if (pdes[0] != STDIN_FILENO) {
                        error = posix_spawn_file_actions_adddup2(&file_action_obj, pdes[0], STDIN_FILENO);
                        if (error) {
                                goto fail;
                        }
                        error = posix_spawn_file_actions_addclose(&file_action_obj, pdes[0]);
                        if (error) {
                                goto fail;
                        }
                }
        }
        (void)__readlockenv();
        error = posix_spawn(&pid, _PATH_BSHELL, &file_action_obj, 0, __UNCONST(argp), environ);
        if (error) {
                (void)__unlockenv();
                goto fail;
        }
        (void)__unlockenv();
        error = posix_spawn_file_actions_destroy(&file_action_obj);
        /*
         * TODO: if _destroy() fails we have to go on, otherwise we
         * leak the pid.
         */
        if (error) {
                errno = error;
                return -1;
        }
        return pid;

fail:
        errno = error;
        posix_spawn_file_actions_destroy(&file_action_obj);
        return -1;
}

On a high level what happens in pdes_child() and popen is that we first lock the pidlist_mutex. Then we create a file file action list for all concurrent popen() / popenve() instances and the side of the pipe not necessary, and the move to stdin/stdout. We unlock the pidlist_mutex. Finally we return the list and destroy.

In the new version of this helper function which now handles the majority of what popen/popenve did, we have to initialize a file_actions object which by default contains no file actions for posix_spawn() to perform. Since we have to have error handling and a common return value for the functions calling pdes_child() and deconstruction, we make use of goto in some parts of this function.

The close() and dup2() actions now get replaced by corresponding file_actions syscalls, they are used to specify a series of actions to be performed by a posix_spawn operation.

After this series of actions, we call _readlockenv(), and call posix_spawn with the file_action object and the other arguments to be executed. If it succeeds, we return the pid of the child to popen, otherwise we return -1, in both cases we destroy the file_action object before we proceed.

In popen and popenve our code has been reduced to just the 'pid == -1' branch, everything else happens in pdes_child() now.

After readlockenv we call pdes_child and pass it the command to execute in the posix_spawn'd child process; if pdes_child returns -1 we run the old error handling code. Likewise for popenve.

popen, old:

FILE *
popen(const char *cmd, const char *type)
{
        struct pid *cur;
        int pdes[2], serrno;
        pid_t pid;

        _DIAGASSERT(cmd != NULL);
        _DIAGASSERT(type != NULL);

        if ((cur = pdes_get(pdes, &type)) == NULL)
                return NULL;

        MUTEX_LOCK();
        (void)__readlockenv();
        switch (pid = vfork()) {
        case -1:                        /* Error. */
                serrno = errno;
                (void)__unlockenv();
                MUTEX_UNLOCK();
                pdes_error(pdes, cur);
                errno = serrno;
                return NULL;
                /* NOTREACHED */
        case 0:                         /* Child. */
                pdes_child(pdes, type);
                execl(_PATH_BSHELL, "sh", "-c", cmd, NULL);
                _exit(127);
                /* NOTREACHED */
        }
        (void)__unlockenv();

        pdes_parent(pdes, cur, pid, type);

        MUTEX_UNLOCK();

        return cur->fp;
}

popen, new:

FILE *
popen(const char *cmd, const char *type)
{
        struct pid *cur;
        int pdes[2], serrno;
        pid_t pid;

        _DIAGASSERT(cmd != NULL);
        _DIAGASSERT(type != NULL);

        if ((cur = pdes_get(pdes, &type)) == NULL)
                return NULL;

        MUTEX_LOCK();
        (void)__readlockenv();
        pid = pdes_child(pdes, type, cmd);
        if (pid == -1) {
                /* Error. */
                serrno = errno;
                (void)__unlockenv();
                MUTEX_UNLOCK();
                pdes_error(pdes, cur);
                errno = serrno;
                return NULL;
                /* NOTREACHED */
        }
        (void)__unlockenv();

        pdes_parent(pdes, cur, pid, type);

        MUTEX_UNLOCK();

        return cur->fp;
}

popenve, old:

FILE *
popenve(const char *cmd, char *const *argv, char *const *envp, const char *type)
{
        struct pid *cur;
        int pdes[2], serrno;
        pid_t pid;

        _DIAGASSERT(cmd != NULL);
        _DIAGASSERT(type != NULL);

        if ((cur = pdes_get(pdes, &type)) == NULL)
                return NULL;

        MUTEX_LOCK();
        switch (pid = vfork()) {
        case -1:                        /* Error. */
                serrno = errno;
                MUTEX_UNLOCK();
                pdes_error(pdes, cur);
                errno = serrno;
                return NULL;
                /* NOTREACHED */
        case 0:                         /* Child. */
                pdes_child(pdes, type);
                execve(cmd, argv, envp);
                _exit(127);
                /* NOTREACHED */
        }

        pdes_parent(pdes, cur, pid, type);

        MUTEX_UNLOCK();

        return cur->fp;
}

popenve, new:

FILE *
popenve(const char *cmd, char *const *argv, char *const *envp, const char *type)
{
        struct pid *cur;
        int pdes[2], serrno;
        pid_t pid;

        _DIAGASSERT(cmd != NULL);
        _DIAGASSERT(type != NULL);

        if ((cur = pdes_get(pdes, &type)) == NULL)
                return NULL;

        MUTEX_LOCK();
        pid = pdes_child(pdes, type, cmd);
        if (pid == -1) {
                /* Error. */
                serrno = errno;
                MUTEX_UNLOCK();
                pdes_error(pdes, cur);
                errno = serrno;
                return NULL;
                /* NOTREACHED */
        }

        pdes_parent(pdes, cur, pid, type);

        MUTEX_UNLOCK();

        return cur->fp;
}
NetBSD.org pkgsrc-2020Q2 released

June 27, 2020

DragonFly BSD Digest In Other BSDs for 2020/06/27

Overflow again, finally.


June 20, 2020

Benny Siegert Getting Started with NetBSD on the Pinebook Pro
If you buy a Pinebook Pro now, it comes with Manjaro Linux on the internal eMMC storage. Let’s install NetBSD instead! The easiest way to get started is to buy a decent micro-SD card (what sort of markings it should have is a science of its own, by the way) and install NetBSD on that. On a warm boot (i.e. when rebooting a running system), the micro-SD card has priority compared to the eMMC, so the system will boot from there.

June 18, 2020

Emile Heitor Fakecracker: NetBSD as a Function Based MicroVM
In November 2018 AWS published an Open Source tool called Firecracker, mostly a virtual machine monitor relying on KVM, a small sized Linux kernel, and a stripped down version of Qemu. What baffled me was the speed at which the virtual machine would fire up and run the service. The whole process is to be compared to a container, but safer, as it does not share the kernel nor any resource, it is a separate and dedicated virtual machine.

June 16, 2020

Stack Overflow screen fails on NetBSD, reporting "poll: Invalid argument"

I have installed and used screen many times on several different operating systems. Recently I installed it on a NetBSD-8.0 virtual machine.

$ sudo pkgin install screen
calculating dependencies...done.

1 package to install:
  screen-4.8.0nb1

0 to refresh, 0 to upgrade, 1 to install
0B to download, 1098K to install

proceed ? [Y/n] Y
installing screen-4.8.0nb1...
screen-4.8.0nb1: setting permissions on /usr/pkg/bin/screen-4.8.0 (o=root, g=wheel, m=4511)
screen-4.8.0nb1: adding /usr/pkg/bin/screen to /etc/shells
screen-4.8.0nb1: registering info file /usr/pkg/info/screen.info
===========================================================================
$NetBSD: MESSAGE,v 1.5 2005/12/28 17:53:24 reed Exp $
[snip]
===========================================================================
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...
marking screen-4.8.0nb1 as non auto-removable

However, when I went to use it, I got an immediate failure.

$ uname -mrs
NetBSD 8.0 amd64
$ ls -l /usr/pkg/bin/screen
lrwxr-xr-x  1 root  wheel  12 Apr  6 02:50 /usr/pkg/bin/screen -> screen-4.8.0
$ groups
users wheel
$ screen
poll: Invalid argument

This problem persists even when I first remove, then reinstall the screen package. Any suggestions as to what's wrong?


June 06, 2020

Frederic Cambus OpenBSD framebuffer console and custom color palettes

On framebuffer consoles, OpenBSD uses the rasops(9) subsystem, which was imported from NetBSD in March 2001.

The RGB values for the ANSI color palette in rasops have been chosen to match the ones in Open Firmware, and are different from those in the VGA text mode color palette.

Rasops palette:

Rasops palette

VGA text mode palette:

VGA text mode palette

As one can see, the difference is quite significant, and decades of exposure to MS-DOS and Linux consoles makes it quite difficult to adapt to a different palette.

RGB values for the ANSI color palette are defined in sys/dev/rasops/rasops.c, and here are the proper ones to use to match the VGA text mode palette:

#define	NORMAL_BLACK	0x000000
#define	NORMAL_RED	0xaa0000
#define	NORMAL_GREEN	0x00aa00
#define	NORMAL_BROWN	0xaa5500
#define	NORMAL_BLUE	0x0000aa
#define	NORMAL_MAGENTA	0xaa00aa
#define	NORMAL_CYAN	0x00aaaa
#define	NORMAL_WHITE	0xaaaaaa

#define	HILITE_BLACK	0x555555
#define	HILITE_RED	0xff5555
#define	HILITE_GREEN	0x55ff55
#define	HILITE_BROWN	0xffff55
#define	HILITE_BLUE	0x5555ff
#define	HILITE_MAGENTA	0xff55ff
#define	HILITE_CYAN	0x55ffff
#define	HILITE_WHITE	0xffffff

And here is a diff doing just that, which I sent to [email protected] back in January 2017.

EDIT: The enthusiasm around this article led me to make another try, which didn't fare any better.


May 31, 2020

Benny Siegert Pinebook Pro, First Impressions
Note: This post was written on the Pinebook Pro :) After seeing it in action at FOSDEM (from afar, as the crowd was too large), I decided to buy a Pinebook Pro for personal use. From the beginning, the intention was to use it for pkgsrc development, with NetBSD as the main OS. It was finally delivered on Thursday, one day earlier than promised, so I thought I would write down my first impressions.

May 27, 2020

Frederic Cambus OpenBSD/armv7 on the CubieBoard2

I bought the CubieBoard2 back in 2016 with the idea to run OpenBSD on it, but because of various reliability issues with the onboard NIC, it ended up running NetBSD for a few weeks before ending up in a drawer.

Back in October, Mark Kettenis committed code to allow switching to the framebuffer "glass" console in the bootloader on OpenBSD/armv7, making it possible to install the system without using a serial cable.

>> OpenBSD/armv7 BOOTARM 1.14
boot> set tty fb0
switching console to fb0

This prompted me to plug the board again, and having support for the framebuffer console is a game changer. It also allows running Xenocara, if that's your thing.

Here is the output of running file on executables:

ELF 32-bit LSB shared object, ARM, version 1

And this is the result of the md5 -t benchmark:

MD5 time trial.  Processing 10000 10000-byte blocks...
Digest = 52e5f9c9e6f656f3e1800dfa5579d089
Time   = 1.340000 seconds
Speed  = 74626865.671642 bytes/second

For the record, LibreSSL speed benchmark results are available here.

System message buffer (dmesg output):

OpenBSD 6.7-current (GENERIC) #299: Sun May 24 18:25:45 MDT 2020
    [email protected]:/usr/src/sys/arch/armv7/compile/GENERIC
real mem  = 964190208 (919MB)
avail mem = 935088128 (891MB)
random: good seed from bootblocks
mainbus0 at root: Cubietech Cubieboard2
cpu0 at mainbus0 mpidr 0: ARM Cortex-A7 r0p4
cpu0: 32KB 32b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu0: 256KB 64b/line 8-way L2 cache
cortex0 at mainbus0
psci0 at mainbus0: PSCI 0.0
sxiccmu0 at mainbus0
agtimer0 at mainbus0: tick rate 24000 KHz
simplebus0 at mainbus0: "soc"
sxiccmu1 at simplebus0
sxipio0 at simplebus0: 175 pins
sxirtc0 at simplebus0
sxisid0 at simplebus0
ampintc0 at simplebus0 nirq 160, ncpu 2: "interrupt-controller"
"system-control" at simplebus0 not configured
"interrupt-controller" at simplebus0 not configured
"dma-controller" at simplebus0 not configured
"lcd-controller" at simplebus0 not configured
"lcd-controller" at simplebus0 not configured
"video-codec" at simplebus0 not configured
sximmc0 at simplebus0
sdmmc0 at sximmc0: 4-bit, sd high-speed, mmc high-speed, dma
"usb" at simplebus0 not configured
"phy" at simplebus0 not configured
ehci0 at simplebus0
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1
ohci0 at simplebus0: version 1.0
"crypto-engine" at simplebus0 not configured
"hdmi" at simplebus0 not configured
sxiahci0 at simplebus0: AHCI 1.1
scsibus0 at sxiahci0: 32 targets
ehci1 at simplebus0
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00 addr 1
ohci1 at simplebus0: version 1.0
"timer" at simplebus0 not configured
sxidog0 at simplebus0
"ir" at simplebus0 not configured
"codec" at simplebus0 not configured
sxits0 at simplebus0
com0 at simplebus0: ns16550, no working fifo
sxitwi0 at simplebus0
iic0 at sxitwi0
axppmic0 at iic0 addr 0x34: AXP209
sxitwi1 at simplebus0
iic1 at sxitwi1
"gpu" at simplebus0 not configured
dwge0 at simplebus0: address 02:0a:09:03:27:08
rlphy0 at dwge0 phy 1: RTL8201L 10/100 PHY, rev. 1
"hstimer" at simplebus0 not configured
"display-frontend" at simplebus0 not configured
"display-frontend" at simplebus0 not configured
"display-backend" at simplebus0 not configured
"display-backend" at simplebus0 not configured
gpio0 at sxipio0: 32 pins
gpio1 at sxipio0: 32 pins
gpio2 at sxipio0: 32 pins
gpio3 at sxipio0: 32 pins
gpio4 at sxipio0: 32 pins
gpio5 at sxipio0: 32 pins
gpio6 at sxipio0: 32 pins
gpio7 at sxipio0: 32 pins
gpio8 at sxipio0: 32 pins
usb2 at ohci0: USB revision 1.0
uhub2 at usb2 configuration 1 interface 0 "Generic OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci1: USB revision 1.0
uhub3 at usb3 configuration 1 interface 0 "Generic OHCI root hub" rev 1.00/1.00 addr 1
simplefb0 at mainbus0: 1920x1080, 32bpp
wsdisplay0 at simplefb0 mux 1: console (std, vt100 emulation)
scsibus1 at sdmmc0: 2 targets, initiator 0
sd0 at scsibus1 targ 1 lun 0: <SD/MMC, SC64G, 0080> removable
sd0: 60906MB, 512 bytes/sector, 124735488 sectors
uhidev0 at uhub2 port 1 configuration 1 interface 0 "Lenovo ThinkPad Compact USB Keyboard with TrackPoint" rev 2.00/3.30 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub2 port 1 configuration 1 interface 1 "Lenovo ThinkPad Compact USB Keyboard with TrackPoint" rev 2.00/3.30 addr 2
uhidev1: iclass 3/1, 22 report ids
ums0 at uhidev1 reportid 1: 5 buttons, Z and W dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 16: input=2, output=0, feature=0
uhid1 at uhidev1 reportid 17: input=2, output=0, feature=0
uhid2 at uhidev1 reportid 19: input=8, output=8, feature=8
uhid3 at uhidev1 reportid 21: input=2, output=0, feature=0
uhid4 at uhidev1 reportid 22: input=2, output=0, feature=0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
bootfile: sd0a:/bsd
boot device: sd0
root on sd0a (f7b555b0fa0e8c49.a) swap on sd0b dump on sd0b

Sensors output:

$ sysctl hw.sensors
hw.sensors.sxits0.temp0=39.50 degC
hw.sensors.axppmic0.temp0=30.00 degC
hw.sensors.axppmic0.volt0=4.95 VDC (ACIN)
hw.sensors.axppmic0.volt1=0.03 VDC (VBUS)
hw.sensors.axppmic0.volt2=4.85 VDC (APS)
hw.sensors.axppmic0.current0=0.11 A (ACIN)
hw.sensors.axppmic0.current1=0.00 A (VBUS)
hw.sensors.axppmic0.indicator0=On (ACIN), OK
hw.sensors.axppmic0.indicator1=Off (VBUS)

May 20, 2020

Amitai Schlair notqmail 1.08 released

notqmail logo

I’m pleased to announce notqmail 1.08, the latest update to notqmail. It addresses security vulnerabilities in qmail 1.03 reported yesterday by Qualys. As a fork, we’ve inherited qmail’s relatively few bugs; as an active qmail fork, we’ve addressed the vulnerabilities with a timely release; and as the community-driven open-source successor to qmail, we show our work.

The 1.08 release also includes many other small improvements we’ve made since 1.07, including fewer compiler warnings, less dead code, published intent to remove more presumed-dead code, my QMAILREMOTE patch, and our first few unit tests. For more detail, see the notqmail 1.08 release notes.


May 16, 2020

Nikita Gillmann On plant - update 2

Continued musings about plant on a saturday evening, still not 100% verified and opinionated as long as I don't double check with my existing notes. It's a bit chaotic, just taking notes.

So, what exactly do I want?

First it's mostly an experiment at this point.

I'm contributing to Guix, I'm contributing (rarely) to Nix, and I'm a pkgsrc developer. I've used and packaged for various other package managers in the past. And I really am unhappy with all of them, all have nice features, and all make compromises at some points, all make me annoyed when working with them for too long.

I could happily settle for Guix if it wasn't for a few nitpicks where the most unchangable ones can be reduced to code structure and certain parts being unchangable.

Before I dropped/indefinitely paused infotropique/OS core, I reworked my fork of Guix in a way that some issues were addressed, but the amount of work and time to rebase on a newer guix version made me reconsider using guix for small gains in it. I effectively had something which wanted to go in a new direction, but running with an engine I had no full control over. Notes prior to joining NetBSD shifted to the point where I often came back to basically wanting NetBSD but structured differently together with a package manager but not as in the old "this BSD can be dissected like Linux Distributions" mistake.

plant is my ideal package manager, and so is infotropique OS/core constructed from it in templates.

It'd base on MesCC, and some core tools.

As roughly listed here already:

Since the idea is to get to an modular extensible approach to package managers, we want the core to be relatively small and where necessary you should be able to replace the code.

It seems to point at some involvement of Lisp again or one of the few other languages which allow this type of code to be written.

Addendum:

In https://issues.guix.gnu.org/41286#1 which started my more public effort of porting Guix to NetBSD, Ludovic Courtès writes:

Hi Nikita,

Nikita Gillmann skribis:

as mentioned in IRC I have begun porting Guix to NetBSD (with the path taken not yet decided upon, just plain building guix itself for now).

Glibc provides argp. Arguably we don't have to check for argp because Guix targets glibc. But I am quiete certain that there will be people who will attempt to do what I am doing and run into this.

Guix targets glibc-based systems, so as you write, it’s reasonable to assume argp is present.

Also, using AC_CHECK_HEADERS doesn’t achieve anything: it only defines ‘HAVE_ARGP_H’ to zero or one.

Last, I don’t want to discourage anyone from porting, but I also want to be clear about what it entails. I’m strongly in favor of supporting only glibc because: (1) after all, it’s about GNU as a system, and (2) my experience with Nixpkgs is that supporting multiple C libraries is just too much work to maintain good support.

Thus I’m closing for now.

In a chat today with Janneke of GNU Mes I got enough pointers to start with adding support for NetBSD to mes. It's a start, but it will take a long time with everything else going on in my life.


May 15, 2020

Nikita Gillmann On plant - update 1

A quick reflection on what plant wanted to be:

In 2015 I started exploring the reliable creation of a live CD system for an application, which lead to thinking about the concept of an Operating System based on Guix, which rapidly diverged from Guix.

One of the main ideas was to provide building blocks instead of a monolithic approach to the core (you will find that you can modify Guix to a decent amount for an application, but some parts are designed in a way that (last tested in 2017) you can't add/override/etc). Furthermore I knew I wanted to test out ideas which would not work with the way Guix as a group works. While Guix and Nix allow yourself to be pretty independent from a central group, in practice you rely on their flow of work, their decisions, the way they upgrade and their infrastructure. Which isn't bad. I just want to test more decentralization, but made the (decision / mistake) to rely for that part at that time on a network which still needs some tweaks to be ready for this. Furthermore I disagree with some decisions which justify the existence of certain kernel flavors based on interpretation of blobs, licenses, how hardware works, and what some people tell others who ask to run this system (just buy a new/supported device is not an option for many people). Next I dislike the way updates are handled, but we all have to make compromises in what can be maintained, security updates handling, etc. I have my own ideas, which I guess/hope are not that new but just encode certain expectations and contracts. Modularity was also a big goal. Did I mention this should be a generic package manager and also operating system builder through templating and inheritance of the language used (maybe you don't know that Gentoo, Guix, Nix, etc are package managers). And much more.

I am going to extend on these points as I go through old stacks of papers and digital notes I assembled, but not all of them right now. This is the first copy of nightly notes I've started again after a long break on (publicly) thinking about package managers. This one is from 4 days ago. Don't take this as finished, technically correct, or whatever. It's just to try and start documenting more publicly what I do. No concept diagrams for now included. It's mostly written without much editing, and I hope to construct a more long-form technical preview once I get to a point where I can start working on it.

plant should be a set of concepts, protocols, and idealized package guidelines. My version of it would then mean I get to use an implementation of it, for example in Common Lisp. Maye the core could be in C. Maybe it could be entirely Lisp. The goal is just that recipes and binaries can be exchanged. If someone ships an extension to read ebuild-like syntax to build a subset of Gentoo with this? It's okay. But there needs to be a native package format first. I'm okay with NetBSD, so I don't really see a future in building yet another Unix-like system (which doesn't mean that infotropique OS/core is abandoned). When implementations can produce bit by bit the same output for builds, this should work. build-system etc can all be extensons people get however they want them to get (implicit: obtaining and depending on them is managed and documented for packages). I think this is similar to how guix conceptually works, but making all of it hackable (tricky for reprpducibility, so future compromises might apply), without any strong opinion on what plant should look like or run. Even the way you use the network, which network, will be up to you. i2p? tor? "clearnet"? GNUnet? plant doesn't care. Trade recipes, no binary builds on a central server, ... Groups can and probably will form to collaborate, but t really should just be well documented to give everyone what they want it to be. This doesn't have to contradict reproducible builds, it just has to work differently (and makes it harder).

lose notes, incomplete,...:


May 13, 2020

Server Fault Non-standard IP address with dashes

I ran the who command on a shared NetBSD box, and this weird user IP came up:

<redacted> pts/33   May 13 02:13  (XXX.XXX.XXX.XXX)
<redacted> pts/35   May 12 20:59  (202-172-110-147-)
<redacted> pts/36   May  6 20:36  (XXX.XXX.XXX.XXX)

I've never seen an IP like that. Obviously, ping 202-172-110-147- will complain with "Cannot resolve ... (Unknown host)".

There was a similar question posted 7 years ago, which posited that it was a non-standard way of denoting IP ranges, but seeing there's a - at the end of the address, it doesn't seem like a similar thing.


Edit:

I've tried reverse DNS with nslookup 202-172-110-147-, which errors with "** server can't find 202-172-110-147-: NXDOMAIN"

Doing w <user> returns:

9:49AM  up 89 days,  7:46, 1 user, load averages: 0.23, 0.18, 0.17
USER          TTY     FROM              [email protected]  IDLE WHAT
<redacted>    pts/35  202-172-110-147- Tue08PM  4:13

Edit 2: This is on NetBSD, not Linux like I mentioned at the beginning (I thought the box was Linux):

$ uname -rsv
NetBSD 8.1 NetBSD 8.1 (GENERIC) #0: Fri May 31 08:43:59 UTC 2019  [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC

Edit 3: Following @NStorm update, I ran w -n to display network addresses as numbers. I still see the same result

$ w -n <user>
 9:57AM  up 89 days,  7:54, 1 user, load averages: 0.12, 0.12, 0.14
USER          TTY     FROM              [email protected]  IDLE WHAT
<redacted>    pts/35  202-172-110-147- Tue08PM  4:22

May 08, 2020

Nikita Gillmann posix_spawn(3) project for The NetBSD Foundation during Google Summer of Code 2020 accepted!

Hi,

I am going to be writing a bit more about NetBSD in the next months.

I applied for 3 different projects for this years Google Summer of Code at NetBSD, and the one for system(3) and popen(3) (and more) to use posix_spawn(3) got accepted:

   Nikita Gillmann
   Make system(3) and popen(3) use posix_spawn(3) internally

   The library functions popen(3) and system(3) are used to create a new shell process and (in case of popen(3) set up IPC channels to the new process). Currently they are implemented using fork(2), execve(2) and do a bit of astonishing complex internal bookkeeping. It should be possible to simplify both implementations using posix_spawn(3) and associated helper functions.

   Mentors

   Jörg Sonnenberger
   Martin Husemann
   Riastradh

There's the announcement on the TNF Blog:

I've got something else GSoC related I'm looking into to maybe write about later.

My work will be commited to https://n0.is/gsoc2020/ (does not exist right now) and synced to https://github.com/teknokatze/gsoc2020 as well as probably elsewhere (for the main repository work).


May 05, 2020

Unix Stack Exchange NetBSD/FreeBSD and TP-Link TL-WN727N: Second encounter

Moved from SU.
First part at https://superuser.com/questions/975564/netbsd-and-tp-link-tl-wn727n-atheros-ar9271-or-ralink-rt5370

So, I've installed NetBSD 7 and device shown again as ugen(ugein, lol).

ugen0 at uhub4 port 8 
ugen0: Mediatek 802.11 n WLAN, rev 2.01/00, addr 2

Then I'm installed FreeBSD 10.2 and ugen again.

usbconfig gives me ugen4.3: <product 0x7601 vendor 0x148f> at usbus4, cfg=0 md=HOST spd=HIGH(480Mbps) pwr=ON (90ma)

So, what's next? Buying new dongle is a last thing, which I'll make.


May 01, 2020

NetBSD.org New Developer in April 2020

April 21, 2020

Kimmo Suominen Serial console for NetBSD

I’ve been switching my NetBSD machines to using a serial console lately, as it is easier to copy output from it (no more screen shots). I would still need to figure out how to setup rtty1 to log console output, so unexpected output could also be inspected later (hello, panic).

On the NetBSD VM its bootblocks must be reinstalled to select a serial port as the console. I’ve been using the auto option, so I can easily switch between the VGA and serial consoles. I set speed to zero, which will use the speed configured in BIOS. The only available “speed” with qemu is 115200 bps.

installboot -v -o console=auto,speed=0 /dev/rsd0a /usr/mdec/bootxx_ffsv2

In Proxmox a serial port must be added to the VM on the Hardware tab. It is important to leave the Display setting at Default (or VGA) — for some reason input from the serial console will not work in the boot menu otherwise. (It will work in BIOS and at the login prompt.)


  1. Also known as remote-tty in Debian. 


April 16, 2020

Brett Lymn Thomas E. Dickey on NetBSD curses


I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses.  It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match.  I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious.  Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.

The first thing that I would like to mention is that the author seems to be incapable of spelling my name correctly.  It isn't hard, 4 whole letters but the only time that it is spelt correctly is when the text is copied from a NetBSD commit.  If you ever do read this Mr. Dickey, I am not offended, plenty of people struggle with my last name, I am quite used to it.

Before going further, just a bit of background as to how I started working on the NetBSD curses library.  I originally was using a commercial x86 port of System V UNIX on my home PC but started looking elsewhere when I found that the SL/IP implementation was bugged.  At the time Linux had no networking at all but the BSD source code had just been ported to the PC in a project known as 386BSD.  Since this had networking I picked this up and then switched to NetBSD early on.  Being used to the SYSV curses I was disappointed to find that there was no real way of handling cursor and function keys in a terminal independent manner as there is in the SYSV curses.  I developed and submitted a patch to the NetBSD curses library that added the support for the keypad() function and modified getch() to perform the conversion of key sequences into key symbols.  It was on the basis of this patch that I was offered the opportunity to be a NetBSD developer.

When working on NetBSD curses my overall philosophy is to follow The Single UNIX® Specification, Version 2: X/Open Curses, Issue 4 Version 2 
(I will refer to this as SUSv2 from now on) where possible, if a behaviour is undefined then I look to ncurses and match the behaviour there if appropriate on the basis that this normally results in less issues for someone porting an application.  I add ncurses extensions in response to problem reports (PRs).

Let's now have a look at Mr. Dickey's comments (I am tempted to use the word diatribe but that seems a bit strong...).  I won't go over all of them but will pick up the topics that he has mentioned going down his page.

I would like to pick up on the comments about making the window structure opaque.  I don't apologise for this, I believe that the fact that WINDOW * was available for applications to poke at is a violation of software layering and just wrong.  Later in Mr. Dickey's page he takes NetBSD to task over error checking (more on this later) but by making the contents of WINDOW available for the application there is little use in error checking if the application can just write what they want anyway.  It took ncurses until 2007 to opaque WINDOW.  Mr. Dickey makes some ominous sounding statements about adding functions to compensate for the opaquing.  As far as I am concerned the interface to update a window is specified by SUSv2, if there are extensions that need to be added they will be added as they are needed.
Given I made WINDOW opaque nearly 20 years ago, I have yet to see a PR to indicate that more is needed.  The positives of making WINDOW opaque are not only enforcing correct layering of software but it also means that the contents of WINDOW can be modified without requiring a major bump of the library since nothing outside the curses library has access so backward compatibility is maintained.

Making the character string in scanw const just makes sense the function does not need to modify the string so it should be a constant.  The NetBSD project does not need to worry about the compiler objecting because the base compiler works with this statement.

Wide curses was a Google Summer of Code project that was done by Ruibiao Qiu.  I think that this was a great success and added some very useful functionality to NetBSD curses.  It seems that Mr. Dickey is a bit confused by this, in actual fact, he is totally wrong on a number of points:
  1. The non-spacing characters are not stored as a linked list, they are an array
  2. The interface for setting non-spacing characters is anything that uses the complex character type (cchar_t)
He seems a bit mystified by the term non-spacing character, this term comes from SUSv2 described here
The non-spacing characters are dynamically added when required, this was my design as I thought that simply having a fixed array for every character cell in the display was a waste of memory, it would be more efficient to only allocate memory as required for the non-spacing characters.

The function specification for ungetch is ambiguous in SUSv2, NetBSD will accept the output from getch on the basis that the input stream has already been processed and it makes no sense to save the last character in what could be a multi-byte sequence.

As for testing of curses, yes, this is mine.  I am happy that at least some of our curses functions are being tested to ensure that any changes made don't affect the on-screen output and that if there are changes that they are highlighted and can be analysed for correctness.  It is too easy with terminal output to have something that "looks right" but is doing the wrong thing in the background.

Now we get to something I wrote on a mailing list:
 
Re: curses: create panel from stdscr?

I stand by my comments there.  I was referring to my attempt to report what I thought was an issue with the ncurses libmenu implementation.  I no longer have the email nor the reply from Mr. Dickey but my clear recollection of the tone and attitude was that I would not be bothered raising issues about ncurses with Mr. Dickey ever again.  I wrote the libmenu and libform libraries for NetBSD using the book "Programmer's Guide: Character User Interface (FMLI and ETI)", this is UNIX System Laboratories book so I regard it as definitive for the libraries.  I noticed a variance in the ncurses libmenu implementation but as I have stated my efforts of reporting that were not fruitful.  Mr. Dickey did reach out to me in late 2017 asking for the email I was referencing, his comment was that my response was vague, actually, my exact words were:
I will try and dig it up.  It was a very long time ago.  I think it was
related to some behaviours in libmenu.  I will try and dig back through
my mail history but it may take a bit of time.
Alas I never found the original email but looking at my commits around libmenu I would have sent that email in 1999 some time so I don't think that this is very surprising.  At the time of the reply I thought that perhaps there was some interest in actually addressing the issue but the actual fact is that all it was was a fishing expedition to provide more ammunition for an attack on NetBSD curses which is disappointing.

Getting back to the email quoted above, in the book I have create_panel() is documented as taking a WINDOW as parameter.  Though, internally, stdscr is treated as a window the books description of create_panel() does make it clear that WINDOW is expected to be the return from newwin() not just stdscr.

As for missing functions, they will be added as and when the developers have time.  A PR may guide us in what is needed first but given that NetBSD is a volunteer project the time that the developers can devote to adding new features can be variable.  We are aiming to have something that is useful and not necessarily a clone of ncurses.

Mr. Dickey makes a point about portability.  I myself have never been concerned about making NetBSD curses portable, this is not an aim that I have ever had.  I would much rather be adding new features and fixing bugs for NetBSD than struggling with the complexities of making the NetBSD curses library portable.  If others wish to do this and feed back fixes then they will be considered but in the main, portability is not a focus.

He also turns on some random gcc flags and then berates NetBSD curses for producing extra warnings.  It is difficult to determine what flags he has turned on to produce these warnings.  The NetBSD build system treats any warnings as errors and the build terminates, the NetBSD curses library builds with the standard NetBSD gcc options without issue.  Without knowing what options were uses and what defines were in effect the information that Mr. Dickey has presented is useless.  He is welcome to submit a PR using the public interface and it will be looked into.

He then notes the output differences in the wide character implementations.  Unfortunately he has not reached out to me (at least) with this information.  It would be useful to have the tools to test this issue and rectify it.  I will note that he used a rather old version of NetBSD, it would be interesting to see if these issues still exists.

We then have a snide little bit about the number of error returns and error checking.  He says "Error-checking in NetBSD curses is seen as a problem by its developers" which is absolutely untrue and uncalled for.  He seems to be forgetting at this by that, by his own admission, that NetBSD curses does not have all the functions that ncurses has so there should be less returns of status simply from that.  Even if this were not true, if we look at the ratio of ERR/OK returns per number of statements - using the numbers from his page - NetBSD curses has 12987 statements which translates to a ERR/OK return every 25 statements where as ncurses has 24566 statements which translates to a ERR/OK return ever 38 statements.  So NetBSD curses actually assigns status codes more often than ncurses.  Regardless, I view this metric as specious - simply counting returns is rubbish, look at this:

if ((x > maxx) || (y > maxy))
         return ERR;

VS:

if (x > maxx)
        return ERR;

if (y > maxy)
        return ERR;


Both do the same thing but we get double the ERR returns in the second one.  I am not saying this is what is happening but just pointing out the fallacy of counting ERR/OK assignments and trying to draw a conclusion from it without properly analysing the code.

He lambasts NetBSD for commenting out unsupported functions in pycurses, this is just a reflection of the NetBSD curses not having all the functions at the moment.  This is a pkgsrc thing and locally applied, pycurses is unique in that it wants all the curses functions so it can provide bindings, since NetBSD curses doesn't have them then the unsupported ones are commented out.  It would be ideal if they all existed, hopefully one day they will.

The final thing I would like to pick up on is the comment:

"Without documentation, there is no point in making bug reports."

Which is just a lame excuse, even the lack of documentation is something that can be a bug report.  There is a public interface for reporting bugs.  Not using it and then complaining on your own website is not helpful to NetBSD but then again, perhaps that is the intention after all.

April 06, 2020

NetBSD.org pkgsrc-2020Q1 released

April 02, 2020

NetBSD.org NetBSD 8.2 released
NetBSD.org Extended support of the netbsd-7 branch

March 20, 2020

Unix Stack Exchange How to change the font size in NetBSD (TTY no GUI)

NetBSD 7.1 (GENERIC.201703111743Z) amd64

In TTY without gui , the better screen resolution is set at boot time but the font is really to small.
So how to change that

On debian dpkg-reconfigure console-setup
in grub GRUB_GFXMODE=800x600

I find wsconscfg with a example :

wsconscfg -t 80x50 -e vt100 3

but I get

wsconscfg: WSDISPLAYIO_ADDSCREEN: Device not configured

I look at man wsconscfg wsconsctl wscons wsdisplay vga etc..
All of them link to the other but I can't find a way.


March 17, 2020

Super User Most Windows-like text editor that can be used through PuTTY in NetBSD?

Please help me find an editor that behaves more like the modern Windows editors do, that I could use on a NetBSD box through PuTTY.

This includes the absense of the distinction between view/insert/append modes, basic shortcuts such as Ctrl+Left/Right to jump a word at a time, and other niceties such as perhaps Home/End and PageUp/Down moving the cursor around (in my vi they just ding).

I can live without macros because I only ever tweak an occasional setting this way, and nothing else.

pico is pretty close, but I was wondering if there are other editors that might fit this bill?


March 15, 2020

Unix Stack Exchange pkgin installation problem (NetBSD)

I just installed NetBSD 7.1.1 (i386) on my old laptop.

During the installation, I could not install pgkin (I don't know why), so I skipped it and now I have a NetBSD 7.1.1 installed on my laptop without pkgin.

My problem is "How to install pkgin on NetBSD (i386) ?"

I found this Unixmen tutorial and I followed it:

I tried :

#export PKG_PATH="http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/6.0_BETA_current/All/"
# pkg_add -v pkgin

And I got :

pkg_add: Can't process ftp://ftp.netbsd.org:80/pub/pkgsrc/packages/NetBSD/amd64/6.0_BETA_current/All/%0d/pkgin*: Not Found
pkg_add: no pkg found for 'pkgin',sorry.
pkg_add: 1 package addition failed

I know this is a wrong command because this ftp address is for amd64 while my laptop and this NetBSD is i386. (I can't find the correct command for i386 )

I also followed instructions of pkgin.net, and I did

git clone https://github.com/NetBSDfr/pkgin.git

on another computer and copied the output (which is a folder name pkgin) to my NetBSD (my NetBSD doesn't have 'git' command)

and then I did :

./configure --prefix=/usr/pkg --with-libraries=/usr/pkg/lib --with-includes=/usr/pkg/include

and then :

make

but I got :

#   compile  pkgin/summary.o
gcc -O2    -std=gnu99    -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-sign-compare  -Wno-traditional  -Wa,--fatal-warnings -Wreturn-type -Wswitch -Wshadow -Werror    -DPKGIN_VERSION=\""0.9.4 for NetBSD-7.1.1 i386"\" -DNETBSD  -g -DLOCALBASE=\"/usr/local\"           -DPKG_SYSCONFDIR=\"/usr/local/etc\"         -DPKG_DBDIR="\"/var/db/pkg\""           -DDEF_LOG_DIR="\"/var/db/pkg\""         -DPKGIN_DB=\"/var/db/pkgin\"            -DPKGTOOLS=\"/usr/local/sbin\" -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_LARGE_FILES -DCHECK_MACHINE_ARCH=\"i386\" -Iexternal -I. -I/usr/local/include  -c    summary.c
*** Error code 1

Stop.
make: stopped in /root/pkgin

I think this error occurs because of the dependencies. (which is mentioned in pkgin.net) but still, don't know how to install those dependencies.

EDIT: I found "http://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/i386/7.1.1/All/" but it still says

no pkg fond for 'pkgin', sorry

SOLVED:

** I solved the problem by writing 7.1 instead of 7.1.1**


March 06, 2020

Stack Overflow How do I set root password for mysql 5.7 on netbsd

I'm trying to get mySQL 5.7 working on NetBSD (unix). When I try "mysql -u root" I get

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

When I try "mysqld --initialize-insecure" I get

[ERROR] --initialize specified but the data directory has files in it.

What do I do?


March 02, 2020

Emile Heitor Let's Encrypt certificates using LEGO

This post is more like a self-reminder on how I setup automatic SSL/TLS certificate renewal on my servers.

I chose LEGO to handle my certificates renewal with Let’s Encrypt because it’s simple to use, has no dependency, great documentation and is worked on at a constant pace.

I found this and this articles very useful, but they are outdated in their use of the tls and http parameters. So here are my notes.

This procedure is Debian GNU/Linux based but I also used it pretty much as-is on NetBSD and FreeBSD, only nginx related PATHs changed.

nginx

In /etc/nginx/sites-available/default:

1
include /etc/nginx/sites-available/letsencrypt;

Which contains:

1
2
3
4
location /.well-known/acme-challenge {
proxy_pass http://127.0.0.1:8181;
proxy_set_header Host $host;
}

Check nginx.conf syntax then reload it.

1
2
$ sudo nginx -t
$ sudo nginx -s reload

Let’s Encrypt

Go to a writable directory where LEGO will write the challenges

1
$ cd ~/www/letsencrypt

To request and install a new certificate, first prepare nginx.conf:

/etc/nginx/sites-available/https:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
listen443 ssl;
listen[::]:443 ssl;
server_namekorriban.imil.net;

ssl_certificate/usr/local/etc/letsencrypt/certificates/korriban.imil.net.crt;
ssl_certificate_key/usr/local/etc/letsencrypt/certificates/korriban.imil.net.key;

# ...

# for future use of the tls challenge
location /.well-known/acme-challenge {
proxy_pass http://127.0.0.1:4444;
proxy_set_header Host $host;
}

# ...

Then execute LEGO with desired parameters:

1
$ sudo lego --email="[email protected]" --domains="korriban.imil.net" --http.port :8181 --http --path=/usr/local/etc/letsencrypt run

And check / reload nginx.

If you need a multi-domain certificate, simply add multiples --domains.

To renew the certificate using the tls challenge if it expires within 30 days:

1
$ sudo /usr/local/bin/lego --email="[email protected]" --domains="korriban.imil.net" --tls.port :4444 --tls --path=/usr/local/etc/letsencrypt renew --days 30

Automatic renewal

1
$ cat bin/lerenew.sh
1
2
3
4
5
#!/bin/sh

cd /home/imil/www/letsencrypt
/usr/local/bin/lego --email="[email protected]" --domains="korriban.imil.net" --tls.port :4444 --tls --path=/usr/local/etc/letsencrypt renew --days 30
nginx -s reload

cron

1
00 3 * * * /home/imil/bin/lerenew.sh >/home/imil/log/lerenew.log 2>&1

reload mail system

My mail system also uses Let’s Encrypt certificates, it is hosted on virtual machines in the same subnet as the web server, which exports /usr/local/etc/letsencrypt/ via NFS. The mail server runs NetBSD. In order to watch any modifications in the certificates directory, I use direvent, which will call a script to reload both dovecot and postfix:

1
$ cat /usr/pkg/etc/direvent.conf
1
2
3
4
5
6
7
8
9
10
11
syslog {
facility local0;
tag "direvent";
print-priority yes;
}

watcher {
path /usr/local/etc/letsencrypt/certificates;
event write;
command "/home/imil/bin/mailrestart.sh";
}
1
$ cat bin/mailrestart.sh
1
2
3
4
5
6
#!/bin/sh

/usr/sbin/postfix reload
/usr/pkg/bin/doveadm reload

su imil -c 'echo "done"|mail -s "postfix and dovecot reloaded" imil'

direvent is started from /etc/rc.local as there’s no init script given with the package.

1
/usr/pkg/bin/direvent /usr/pkg/etc/direvent.conf

February 17, 2020

OS News NetBSD 9.0 released

The NetBSD Project is pleased to announce NetBSD 9.0, the seventeenth major release of the NetBSD operating system. This release brings significant improvements in terms of hardware support, quality assurance, security, along with new features and hundreds of bug fixes.

Support for the ARM architecture seems to be a major pillar of this new release.


February 13, 2020

Unix Stack Exchange Difference between `pkgin` and `pkg_*`, and which to use?

Coming from Linux land, I assumed that pkgin is some sort of higher-level frontend for pkg_add et al, something like what apt is to dpkg or yum/dnf to rpm. But pkg_add appears to handle installation from network, dependencies, as well as automatic updates (the things that on Linux would be the responsibility of the front-ends rather than the underlying package tool), so now the roles of pkgin and (what I gather are the more traditional) pkg_ tools seem a bit unclear to me. The only difference that I'm presently aware of is that pkgin does not handle installation from sources.


February 03, 2020

Benny Siegert How to do Pull-ups to pkgsrc-stable
I am part of the pkgsrc releng (release engineering) team. My main task there is handling pull-ups into the stable branch. pkgsrc creates a stable branch every three months and names it after the respective quarter – for example, the last branch was called 2019Q4. Pull-ups are tickets to “pull up” one or more commit from the development branch into the stable branch. Typical justifications for pull-ups are: security updates build fixes important bug fixes (such as when the package crashes on startup) In addition, sometimes we pull up updates to packages if they are “leaves” and stop working without regular updates.
Kimmo Suominen Updated sudo

I updated security/sudo to 1.8.31. A fix for CVE-2019-18634 is included.

Originally only versions before 1.8.26 were thought to be vulnerable. Later analysis, however, showed that versions 1.8.26 – 1.8.30 are also vulnerable.

Here’s what’s new:


January 25, 2020

Stack Overflow Alternatives to Vagrant for OS emulation/development [closed]

I am the maintainer psutil project. Since psutil supports different operating systems, I've been using Vagrant in order to to emulate them and develop on them from my Linux box. This worked flawlessly for many years (due to Vagrantfiles, which are great), but unfortunately I got to a point that Vagrant is no longer usable anymore. Problems derive mostly from the inability to mount shared folders, freezes when connecting via SSH and occasional incompatibilities between Vagrant and VirtualBox after a system upgrade. Are there viable alternatives to Vagrant? My working environment is Ubuntu and the systems I need to emulate are BSD*, MacOS and OpenSolaris (for Windows I use VirtualBox directly).


January 24, 2020

Kimmo Suominen Added patches to libxml2

I added patches to textproc/libxml2 from an upstream commit and an upstream pull request to address CVE-2020-7595 and CVE-2019-20388 respectively. Version 2.9.10nb1 includes the patches.


January 19, 2020

Kimmo Suominen Patch for HTTP Request Smuggling

I added a patch to www/nginx from an upstream commit to address CVE-2019-20372. Version 1.16.1nb2 includes the patch.


January 11, 2020

Kimmo Suominen Cherry-picked patches for ncurses

In order to reduce the number of vulnerabilities on my systems, I added some patches to devel/ncurses (and devel/ncursesw) to address CVE-2018-19211, CVE-2019-17594, and CVE-2019-17595. Version 6.1nb7 includes the patches.