Using Splunk as a syslog replacement

Many Splunk users find themselves standing up a “syslog box,” for the purpose of receiving syslog inputs, writing them to disk, running a universal forwarder, monitoring those files that you just created, reading in those files, then sending the data off to the indexer cluster. Why not cut out the middlemen and use Splunk as a direct syslog replacement? It’s not dead-simple, especially in a clustered environment, but it’s not TOO hard…

Continue reading “Using Splunk as a syslog replacement”
Using Splunk as a syslog replacement

The Job Description, the Whole Job Description, and Nothing But the Job Description

Words mean things. Especially words in job descriptions, and even when those words are missing.

I recently did a couple of interviews for a large web hosting company – big enough that if you know the space, you’d know them. Here’s a description of their requirements and expectations for a recent SRE (site reliability engineer) opportunity they posted:

  • Knowledge of large-scale, high traffic platforms and the design of scalable, robust services in the real world
  • Experience with Infrastructure as code tooling (e.g., Terraform, Chef, Puppet, Ansible, Pulumi, Vault, etc)
  • Experience programming in one or more of the following: Go, Python, Ruby, C, C++, Java, etc
  • Experience with Unix/Linux operating systems internals (e.g., filesystems, system calls, namespaces, containers)
  • Knowledge of large-scale, high traffic platforms and the design of scalable, robust services in the real world
  • Experience with analyzing and troubleshooting systems.
  • Understanding of standard networking protocols and components such as: TCP/IP, HTTP, DNS, IP Subnetting, and Load Balancing
  • Passion, integrity, and humor that makes our team better as a whole

This is a very reasonable-looking job description! I’ve built large, scalable platforms, though nothing quite as large as what they’ve built. I’ve used Puppet and Ansible. I’ve done some programming, though in fairness mostly smaller projects, and I haven’t touched C or Python much in a while. I’ve built big servers, I’ve built big networks, I know a fair bit about *nix internals. I know my way around tcpdump and wireshark, I can troubleshoot network protocols with the best of ’em. I also had most of the nice-to-haves. (One of the nice-to-haves was “polyglot chops,” listing off several programming languages they use. But it was a nice-to-have, so I wasn’t too worried about its absence.)

Based on the above, I felt pretty darned confident going into a Zoom interview with a hiring manger. (I did the initial HR screen a few days previous.)

Almost immediately I was asked about the programming languages I’ve worked with. Like, within maybe five minutes. We were barely past the small-talk stage when my development experience (of which there is precious little, which is why it’s not on my resume) came up. When I said that I know some basic stuff but haven’t done a great deal of programming in the last several years, and most of what I’ve done was fairly domain-specific, you could almost see her heart break a little.

Turns out, the job would be about 50 percent SRE (the thing they advertised), and 50 percent developer. Which I’m not.

Fortunately, all this intervew meant was a couple hours of wasted time on my part (and on theirs), I don’t desperately need another job at this time. But if they had listed all of their requirements, and been a bit more up-front and accurate about the role, I could have saved everyone involved some time.

If you’re writing a job description, please include the whole thing, not just half of it.

The Job Description, the Whole Job Description, and Nothing But the Job Description

Clear Linux Cheat Sheet

Clear Linux has Big Slackware Energy, in that if you want to use it as a daily driver, you’ll need to know your way around the underlying software. That said, I don’t think that’s how you’re supposed to use it (despite what that one Phoronix post might have you believe). It really feels purpose-built for connecting to cloud-init or Ansible or something like that, for machines you’ll spin up then throw away tomorrow afternoon.

Things I’ve learned while tinkering with Intel’s new Clear Linux distribution. Will probably get updated from time to time, unless I get bored.

  • Checking for updates: sudo swupd update
  • Installing a new bundle (their version of a package group): sudo swupd bundle-add bundlename
    • Bundles you’ll probably want for general tinkering: dev-utils, tmux
    • Bundles you might want if you want a traditional GUI experience: desktop (GNOME) or maybe os-utils-gui (a basic xfce desktop)
  • Growing the hard disk (probably needed if you want to install GNOME on a VM): see this (basically, grow it the VM-specific way, install the storage-utils bundle, use parted and resize2fs to actually use the new space)


Clear Linux Cheat Sheet

PAC Files and the Inconstant Constant

Recently, I was working on an update for the PAC files we use at work. Had a clever idea for making clients more resilient in the event of an individual proxy failing – just list all the proxies in the PAC file, and use a bit of MATH to spread the load. This initially failed, but not for any of the reasons you might expect…

Continue reading “PAC Files and the Inconstant Constant”
PAC Files and the Inconstant Constant

Updating your WSL Debian Image to “Buster”

While Windows Subsystem for Linux (WSL) isn’t perfect, it’s coming along nicely as a replacement for many of the Unix-y things I used to do with Cygwin. And since most of my server work involves Linux as opposed to generic *nix, it’s nice to have handy access to something that more closely resembles my work environment.

Just a few days ago, the Debian project announced “Buster,” the first major upgrade in a couple years. (If you want to be boring, it’s also “Debian 10.0”.) As of this writing, the Microsoft app store doesn’t have Buster ready-to-go. And even if it did, there’s not a super-clean way to migrate any custom work, home directories, etc. from an old Debian installation to a new one. So here, I throw caution to the wind and update my existing Debian WSL installation from “Stretch” (9.x) to Buster.

Continue reading “Updating your WSL Debian Image to “Buster””
Updating your WSL Debian Image to “Buster”

Changing the screen resolution on a RHEL guest under Hyper-V

I wanted to tinker with the Red Hat Enterprise Linux 8 beta at work. Since it’s work, I’m technically not supposed to use the free-as-in-beer version of VMware Player, even though it seems like everyone does. And hey, my Windows laptop comes with Hyper-V, which should do the job just fine.

Hyper-V is a younger product, and it seems like it’s not as well-supported. In particular, if I’d spun up a RHEL 8 guest under VMware, I strongly suspect that changing the screen resolution wouldn’t be a problem at all. But Hyper-V only appears to present a single default resolution from its virtual frame buffer to the guest.

Anyway, if you go into the GUI settings on a RHEL 8 guest, I didn’t see any other resolutions available to me, just an odd “1152×864” default. And there’s no obvious way to change that, even though the guest video card is capable of more.

To change how the frame buffer is presented to the guest, you actually have to provide a kernel command-line option. You could probably do this at boot-time via the GRUB interface, but the lazy way to do it is just to use grubby to make the change permanent:

sudo grubby --update-kernel=ALL --args="video=hyperv_fb:1440x900"

You can of course enter any resolution you like. (If you like to full-screen your guests, you might want 1920×1080, or whatever matches your monitor.)

Since this is a kernel option, you can’t just logout and login again; you must reboot before you’ll be able to use the new resolution. And you still won’t be able to resize the guest screen dynamically; any time you want to change it, you’ll have to reboot the guest after. This seems like something that will probably get hammered out by the time RHEL 8 is released.

Changing the screen resolution on a RHEL guest under Hyper-V

How to apply for a nerdy job at Bandcamp

One of the jobs currently posted at Bandcamp is a “Senior Fraud/Risk Engineer”. To apply, you have to finish a small puzzle, to demonstrate your infosec basics. Note that if you have to read a blog post to figure it out, you almost certainly are not qualified for the job. (I’m not qualified for the job either, but I like puzzles.)

Spoilers abound, obviously…

Continue reading “How to apply for a nerdy job at Bandcamp”
How to apply for a nerdy job at Bandcamp