Brain Dump 2022🔗
Looking back on 2022🔗
Old Job🔗
Working in mortgage finance industry showed me the reasons I don't want to work with Windows, or Excel, or mainframes... But it did give me a bunch of Python data pipelining skills and exposure to what it takes to convert an organization to a data warehouse over Postgres.
New Job🔗
Moved into the climatetech / fintech space with Sensible Weather and enjoying contributing to a profit center rather than cost center. Stretching me to learn Golang for web services, which has been a good addition to the toolkit.
Old Blog🔗
To give me confidence in job searching I started a new blog. This version was with Fastpages, which has some special handling for converting jupyter notebooks to Jekyll blog pages.
I converted a few old markdown blog entries and wanted to write new posts for side projects and explorations.
The past years have been a lot about finding what niches of software really excite me (which is a variety!). Python as a backend and data engineering and scripting and pseudocoding language is my current comfort. While a language is only a tool, it's been great to be familiar with it as I've explored Time Series and Geospatial data this year.
New Blog🔗
New Year, New Blog Framework. Going with Mkdocs Material and will likely join the sponsor program to get insider access to the official blog plugin.
Fastpages has a bit too much surrounding setup for me to feel in control and mkdocs has some fantastic support for customization and extension out of the box. I'll probably still use something to convert Jupyter notebooks into markdown, but that's still up in the air.
I'm also interested in a strong search, since I'd still like to accumulate knowledge of frontend topics and general know-how on how to set up and deploy systems in different environments. But I want my blog topics to have a more focused curation. Maybe. Who knows where it'll go in a year.
Old Side Projects🔗
While I'm beginning to appreciate the usecases for splitting webservices up into separate deployments, I think I learned to create applications in a generation of extreme over-engineering. I thought to have a shot at getting a job I had to be able to:
- Write and test a Javascript frontend in React / Vue / Angular
- Write and test a backend web framework to provide a JSON REST(-ish) API that is consumed by the frontend
- Create and maintain a database to provide persistence for the application
- Write and understand a proprietary YAML config description for deploying each piece in a horizontally scalable way
So struggling through side projects came with a lot of cognitive thrashing trying to learn multiple things at once and not a lot of feeling of progress.
New Side Projects🔗
I think I signed up for Streamlit beta after I saw it on reddit, but didn't open it until a mentor suggested I use it for teaching students. After some success with students creating impressive demos, I got used to using it on my own.
A handful of gif demos are on my github page, but the main point is I decided to abandon complexity and focus on learning.
I made an OCR app backed by AWS text services, then later discovered EasyOCR and plugged that in instead. I wouldn't want to make changes like that in a highly complicated project, but when working on one or two files it is easy to experiment.
I started working with more tabular and time series data and explored DuckDB and Apache Arrow, then since joining a climatetech company I've been playing with Spatialite, PostGIS, and GeoPandas.
Old Achievements🔗
I put up 17 blog posts on my Fastpages based blog this past year and 3 videos on Youtube. I shared 13 projects on Twitter and the Streamlit Forum and probably hacked away at a dozen others. I joined the open source Streamlit Creators group and got an article published on the Streamlit blog.
New Achievements🔗
I plan to create more videos this year on Youtube. If short videos seem to help the next generation of engineers I want to see what niches are relying on TikTok.
I want to blog more about geospatial topics as I learn more. Geoparquet and DuckDB-Geospatial support are still in very early days. Any climate or energy or population modeling relies on these kinds of computations.
I hope to stick with this single mkdocs deployment for my blog for at least a year. I do have a fun idea to combine it with a Pocketbase based user / event store for making a lightweight Learning Management System of sorts.
Created: June 7, 2023