Category: Open Source

Adventures in open source technology.

I normally end up doing most of these activities outside of work, on my nights and weekends.

Boot Log Information on Linux

Post author By Vince
Post date 2021-08-02
No Comments on Boot Log Information on Linux

I’ve always struggled to find info about the boot!

What drives are in the system?
What’s the IP address?
What are some details about the hardware?
Etc.

Turns out, it is really easy.

journalctl -b

Tags boot, diagnostics, linux, utilities

Open Source Tech

Use InfluxDB 2 as a Metric Server for Proxmox

Post author By Vince
Post date 2021-05-04
2 Comments on Use InfluxDB 2 as a Metric Server for Proxmox

Architecture

InfluxDB will run on in a Docker container.

This setup assumes that a Docker host is accessible on the LAN. In this example, the local domain is vnet and the host name is docker4.

Therefore, the Docker host FQDN is docker4.vnet

InfluxDB Container

Your USERNAME, PASSWORD, and ORG will vary.

I made my org the same as my local networking domain, but that is not a requirement.

docker run -d \
     -p 8086:8086 \
     --mount type=volume,source=influx-data,target=/var/lib/influxdb2 \
     --mount type=volume,source=influx-conf,target=/etc/influxdb2 \
     --name idb \
     -e DOCKER_INFLUXDB_INIT_MODE=setup \
     -e DOCKER_INFLUXDB_INIT_USERNAME=vince \
     -e DOCKER_INFLUXDB_INIT_PASSWORD=secret-password \
     -e DOCKER_INFLUXDB_INIT_ORG=vnet \
     -e DOCKER_INFLUXDB_INIT_BUCKET=proxmox \

Connect Proxmox to InfluxDB 2

As of Proxmox 6.4, it is possible to use the version 2 API of Influx.

Before that, Proxmox would connect to Influx 1.x, but it was sending graphite line protocol over UDP (yuck). Now it connects via TCP/HTTP to the Influx API.

Get the Token

Log into the Influx web user interface, and copy the token for the user.

Click the token name to reveal the token.

http://docker4.vnet:8086/

There is probably a command-line way to do that, but I don’t know it.

Configure Proxmox Metric Server

The name can be whatever you want.

Paste the token from the last step into the “Token” field in the Proxmox UI.

Confirm Proxmox is Saving to InfluxDB

Navigate back to the InfluxDB web UI, and confirm that data is flowing. http://docker4.vnet:8086/

If things are working, there will be some measurements that clearly look related to Proxmox resource utilization.

Next Steps

In the next post, we will connect the InfluxDB bucket to a pre-built dashboard using Grafana.

Tags api, data, docker, flux, influxdb, influxdb 2.0, proxmox, time series databases

Open Source Tech

Disable Multipath for Identical NVMe Drives on Ubuntu Server

Post author By Vince
Post date 2021-02-18
No Comments on Disable Multipath for Identical NVMe Drives on Ubuntu Server

I have 4 identical NVMe drives installed in my system.

Ubuntu treats them as one “multipath” device by default. This is not accurate.

I posted something on Ubuntu Forums, but no response at time of writing. (2021-02-18)

Here’s the problem:

lsblk    
 nvme0n1                   259:0    0   1.9T  0 disk  
 └─mpatha                  253:1    0   1.9T  0 mpath 
 nvme1n1                   259:1    0   1.9T  0 disk  
 └─mpatha                  253:1    0   1.9T  0 mpath 
 nvme2n1                   259:2    0   1.9T  0 disk  
 └─mpatha                  253:1    0   1.9T  0 mpath 
 nvme3n1                   259:3    0   1.9T  0 disk  
 └─mpatha                  253:1    0   1.9T  0 mpath

The fix is relatively simple. It is outlined in some SUSE Documentation.

#/etc/multipath.conf

blacklist {
       devnode "^nvme[0-9]"
 }

Once the config file has been changed, apply the changes according to the Ubuntu Server Documentation.

sudo systemctl restart multipathd

And all is good!

lsblk
 nvme0n1                   259:0    0   1.9T  0 disk 
 nvme1n1                   259:1    0   1.9T  0 disk 
 nvme2n1                   259:2    0   1.9T  0 disk 
 nvme3n1                   259:3    0   1.9T  0 disk

Tags default, multipath, multipathd, nvme, ubuntu

Open Source Tech

Fail: USB Passthrough to Dexcom 6 Reciver

Post author By Vince
Post date 2021-02-06
No Comments on Fail: USB Passthrough to Dexcom 6 Reciver

My T1 Diabetic Connundrum

I want to connect my Dexcom receiver to a computer. This will allow me to track my blood glucose over time. Then, I can identify trends and patters to help improve my insulin dosing.

Mac and Linux

Dexcom’s software only appears to work for Windows. I only have Mac/Linux/FreeBSD machines at home. I want to pass the USB Dexcom device through to a Windows VM.

Proxmox USB Passthrough: Success

For many people and use cases, this seems to work fine!

I was able to pass a USB flash drive through to a Windows VM with no issue.

Proxmox USB Passthough: Fail

Things were looking promising after the USB flash drive worked!

They looked even more promising after the Qemu Monitor recognized the Dexcom device.

qm> info usbhost
   Bus 1, Addr 15, Port 10, Speed 12 Mb/s
     Class 02: USB device 22a3:0047, DexCom Gen4 USB Serial

Unfortunately, the device was not recognized by the Windows Device Manager.

I thought this might be fine, and that the Dexcom software would find the right driver and solve the problem! However, the Dexcom program gave prompts as if the device was not plugged in.

Troubleshooting

Proxmox has robust documentation on USB passthrough on their wiki.

The basic things I tried were

Physically unplug and re-plug the device
Drop/Add the Qemu USB device using USB bus/port numbers
Drop/Add the Qemu USB device using Vendor and Product IDs

Sample Commands:

#device info
 qm> info usbhost
   Bus 1, Addr 15, Port 10, Speed 12 Mb/s
     Class 02: USB device 22a3:0047, DexCom Gen4 USB Serial
 
#by bus and port
 device_add usb-host,hostbus=1,hostport=10,id=usb1
 device_del usb1

#by vendor and product ID
 device_add usb-host,vendorid=0x22a3,productid=0x0047,id=usb1
 device_del usb1

I tried in different USB ports, after restarting VM, and other miscellaneous troubleshooting as well.

The results were consistently the same failure.

Resolution

I borrowed a Windows laptop, and it worked fine!

I’m not sure what the root cause is for this issue. One possibility is that the Vendor ID seems to be in USB device databases on the web, but the product ID is not.

It is possible that Proxmox needs to have a valid/known product ID to pass to the VM.

It is also possible that the Dexcom software is written in a way that just doesn’t work when the USB device is not on the bare-metal machine. After all, the VM does need a virtualized USB hub etc., so it might just inevitably break some code.

Another possibility is that the Windows OS layer isn’t pulling the device details through properly, even though Proxmox is sending all it can.

Who knows! With such a tall software stack, combined with a tiny user base (type 1 diabetics with this particular device who are tech nerds like me) the answer to this question is likely to remain a mystery forever. Oh well.

Tags CGM, clarity, dexcom 6, diabetes, glucose monitoring, proxmox, usb, virtualization

Open Source Tech

Bulk Resize Images for WordPress

Post author By Vince
Post date 2021-01-04
No Comments on Bulk Resize Images for WordPress

Here’s a python script.

import os
import subprocess

# inspired by this post
# https://guides.wp-bullet.com/batch-resize-images-using-linux-command-line-and-imagemagick/

def main():
    # the maximum length in any direction (width or height)
    # setting at 1600 because full HD (1920x1080) seems larger than necessary
    # this will allow 1600x900 (or portrait 900x1600) images.
    MAX_LEN = "1600" 
    size_parm = MAX_LEN+'x'+MAX_LEN+'>' #https://legacy.imagemagick.org/Usage/resize/#shrink
    
    # files in the input directory will (at least SHOULD) remain untouched.
    # files with identical names will be created in the output directory.
    # files in the output directory are OVERWRITTEN (if they already exist and the names match)
    IN_DIR = '/home/vince/Pictures/WordPress'
    OUT_DIR = '/home/vince/Pictures/WordPressSmall'

    # show what is about to happen
    print ('Input: ', IN_DIR)
    print ('Output:', OUT_DIR)
    print ('Resize Geometry:', size_parm)

    for filename in os.listdir(IN_DIR):
        # get full file paths
        infile =  os.path.join(IN_DIR, filename)
        outfile = os.path.join(OUT_DIR, filename)

        # this is not necessary because the ImageMagick library supports shrinking only
        # AND maintaining aspect ratio. all the logic is basically done.
        # the '>' in size_parm prevents making the image larger.
        # #get width and height
        # width = subprocess.check_output(['identify', '-format', '%w', filepath])
        # width = int(width.decode())

        # convert and print output
        subprocess.check_output(['convert', infile, '-verbose', '-resize', size_parm, outfile])

         
if __name__ == '__main__':
    main()

Tags imagemagick, linux, wordpress

Open Source Tech

Torrent Container for Proxmox

Post author By Vince
Post date 2020-10-31
No Comments on Torrent Container for Proxmox

Why Do This?

I always end up downloading ISO images for use in VMs using my laptop. I’m rarely running the torrent program on my laptop, so I am a leecher!

Proxmox is running 24/7, so I can help the rest of the community by being a seeder!

Creating a container to avoid having too much configuration on the Proxmox server itself. The idea is to keep Proxmox a pure VM host, not a Torrent server.

Plus, the setup is finicky, so being able repeatedly wipe the container and start over is helpful!

Before We Begin

The setup assumes that networking has been configured for both the Proxmox host, and the container that will run the Transmission torrent client.

In case local DNS is not configured, IP addresses are fine!

pve1: Physical server that is running Proxmox
torrent0: Debian container to run Transmission and Samba.

Both of these servers have IP addresses on the 192.168.27.0/24 network.

Transmission (Torrent)

root@torrent0:~# apt install transmission-daemon transmission-cli
root@torrent0:~# cp /etc/transmission-daemon/settings.json /etc/transmission-daemon/settings.json.original

root@torrent0:~# service transmission-daemon stop

root@torrent0:~# nano /etc/transmission-daemon/settings.json
    "download-dir": "/srv/smb/template/iso",
    "rpc-password": "YourSecretPassword",
    "rpc-whitelist": "192.168.27.*",

root@torrent0:~# service transmission-daemon start

Downloading a torrent will cause that that directory to be automatically created if it doesn’t exist. Avoid doing that until we’ve completed more steps in the setup.

For more information, see Debian’s documentation.

Samba (CIFS / SMB)

root@torrent0:~# apt install samba

root@torrent0:~# smbpasswd -a debian-transmission
root@torrent0:~# mkdir /srv/smb
root@torrent0:~# chown debian-transmission:debian-transmission /srv/smb/

root@torrent0:/etc/samba# cp smb.conf smb.conf.default 
root@torrent0:/etc/samba# nano smb.conf

[smb]
   comment = SMB Share 
   path = /srv/smb 
   guest ok = no
   browseable = yes
   readonly = no

root@torrent0:~# systemctl restart smbd
root@torrent0:~# systemctl status smbd

For more information, see Debian’s documentation.

Connect Proxmox

I had buggy issues removing/editing CIFS shares from the web GUI. Falling back to command line.

root@pve1:~# pvesm add cifs tor0 --server torrent0.vnet --share smb --username debian-transmission --password 'YourSecretPassword'
--then manually edited to change storage type to ISO in GUI

This automatically creates /template/iso/ under whatever share you give it.

For more information, see the Proxmox Storage Documentation

And also, forum post related to bugs adding/removing CIFS.