Recent AI Endeavors

 

Why must we imagine Sisyphus happy? tldr, because he may as well be.

I'm playing around with local AI models again since I recently got a newer, more AI-capable GPU. These newer gen models have gotten more interesting; I can now happily run Qwen3.8-27B Q4 with decent context. Context in this case is just as you'd think, ability to keep context within a conversation. Incidentally, I suspect there are many people out there running in low-context mode, at least sometimes, revealed by just how quickly they seem to forget things when called out on something that doesn't reconcile with past conversations, not to say that I'm beyond it. Keep in mind, the instance in the screenshot is all running on local hardware, no internet required, all in a 16GB file. This size is interestingly less than the whole of wikipedia that you can download for offline viewing which is roughly 50GB-100GB depending on if you want images. I still revel in being able to have the entirely of Wikipedia available to me for offline viewing. I'm like a squirrel with information, I like to harvest for proverbial winters so I can look at it and find comfort in having an independent way of taking care, or at least, entertaining myself.

For some teaching purposes, AI models seem to provide more information than even an offline instance of wikipedia can do unless you count the human ability to extrapolate information from said source. Well, so you say, how can AI models/clients explain more with less data? I'm not an AI expert, I just dabble in it sometimes but as I understand it, roughly through data compression, to be overly reductive. Which I think is kind of evocative because what separates humans from most other high-order animals? One key quality is our ability to compress information which is probably one of the least controversial ways we can define intelligence. We want to fit things in neat little boxes, find what's similar, what's exclusive about them and how the dots can be connected. And here I am about to go off like an expert and elaborate more, mentioning tokens and such when I realize I'm not even sure that I understand that well about how AI does its magic, enough to rightfully teach anybody else. Here, while I have it running, let's be lazy and let our AI overlords explain it:

"Wikipedia's 50GB is mostly raw data: facts, prose, images, links. Much of it is redundant, unstructured, and requires human interpretation to extract meaning. A lot of it is also low-value (typos, editorial chatter, minor edits).

• A 15GB AI model is mostly compressed learned structure: weights that encode statistical patterns, relationships, and abstractions distilled from a much larger corpus. The "information" it carries is:

• Dense and compressed – parameters are learned representations, not stored text. 15GB of weights can implicitly encode knowledge from hundreds of GBs of training text.

• Structured and relational – it knows how concepts connect, not just that they exist. That relational structure is what lets it answer novel questions, generalize, and reason.

• Parameterized for inference – the value is in what it can do (generate, extrapolate, reason), not what it literally contains."

So basically they're trained on so much more data than what's indexed on Wikipedia and data is tokenized (mapped to an integer), weighted (biased according to desired output) and spit out very confidently (and sometimes incorrectly) to (opinion incoming) what can be very unassuming people who may not take the output with a grain of salt as they should. It's come a long way with every iteration I've tried over the years and has become less erroneous but I worry that it will get so good that the less critical among us won't see its faults and assume it can do or know no wrong, as we can tend to do with certain other people, unfortunately. Well, that's a whole other sociopolitical tangent I can go on but I can't help but find analogs between technology, biology, sociology and such with varying degrees of relevance. It comes to me as no surprise, after all, since all scientific disciplines are essentially the study of information and flow thereof. 

Anyway, I didn't intend to write this much but I should know better because when I write I write, while inbetween long periods of not being that expressive. It's now a day after I wrote this post, intending maybe I'd post this on my long, forgotten blog because my recent experience with LLM models and upgraded hardware was notable enough that I wanted to... well, note it. When I first saw the movie AI, I found the Dr Know scene fascinating and realized that's kind of what I've been trying to do all along, in a sense. Create a vast index of information that could easily be retrieved in an instant, a library at the tip of my fingers, just waiting for me to explore. Although thankfully modern AI isn't as commercialized as Dr Know is in the movie but it's still painfully obtuse sometimes and still paid for in one way or another, in this case with hardware, but at least in local LLM's defense, ads aren't fluently built into it yet. 

In keeping with tradition, I'll include some tunes. Post-Soviet syth-pop and darkwave just speak to me sometimes. Between the late Soviet era, namely perestroika, and modern times where creative censorship is once again fashionable in the region, Russia had such a thriving music scene. In it, at least to what genres have attracted me, you can feel the sense of cautious hope, apathy, and desperation that was rampant in the air during those darker times of economic and cultural transition. Alliance's Na Zare is one such song that I only recently discovered but apparently and unsurprisingly popular in late 80s Russia. I once spent an entire one hour motorcycle trip with the song repeating in my head and between the song, the weather, recent events and general vibe I was feeling, it was such a glorious and memorable ride. Apparently the band and song have made a more modern and seemingly meme-worthy resurgence on youtube and rightfully so when you hear the song and see the video. Talk about contrasts, between the singer's haunting and melancholy delivery and the background keyboardist's vibrant presence, it's almost as if they're playing different songs, to steal one commenter's reply, yet it all comes together so beautifully. Case in point: