Hooking an LLM into the Windows Kernel, Console, and Filesystem

1. The Idea: How It All Started

Around early 2023, Generative AI exploded, and tech discussions turned into an endless parade of venture capital hype. Social feeds were dominated by people building "AI-based" SaaS websites, slapping a quick prompt on an API, and calling it a day. I had zero interest in building another corporate productivity tool or spreadsheet assistant. While I found AI to be a genuinely interesting innovation, I wanted to use it in my own way—as a personal project, exploring what the technology could actually do rather than chasing commercial trends.

I heard about the OpenAI API, and that's when the idea clicked: I should build an assistant. But OpenAI was paid and required credit card billing, which was out of reach for a student with no disposable income. So I did some research and came across Google PaLM (which was launched as Bard, and later rebranded to Gemini), which was offering a free API service. I used that to experiment with what I could actually pull off and decided I wanted an assistant who lived on my machine, bantered like a friend, and actually had "hands" on the operating system.

2. How It Began: Beyond Basic Voice Wrappers

When I first looked into assistant tutorials at the time, they were all essentially the same loop: accept voice input → fetch the OpenAI API → generate text → pass it to TTS and call it a day. Nothing else. There was zero interaction with the machine itself—no awareness of running processes, no terminal feedback, and no actual control over the computer.

It's not like the concept of an OS assistant was completely new—Microsoft already tried Cortana (which everyone ended up disabling) and eventually cooked up Recall. But Cortana was just a locked-down Bing search widget, and Recall is corporate telemetry taking constant snapshots into a database. More importantly? Neither of them were fun. I wanted something personal that could actually do what Cortana and Recall both can't: have real "hands" on the operating system, roast me in the terminal, talk to kernel drivers, and feel like a companion I actually had fun building and using.

And so, the project started under the name AIMS which had a full form btw: Artificial Intelligence Made From Scratch. Crazy name for a simple PaLM API wrapper, right? Lmao.

Anyways, the initial prototype was admittedly naive: a Python script running against the PaLM API with a cheerful anime persona, executing commands through crude substring matching:

# The 2023 execution mechanism: substring matching + raw batch scripts
if "genshin" in user_input.lower():
    os.system(r"ai\batch_files\genshin_launch.bat")
elif "debloat" in user_input.lower():
    os.system(r"ai\batch_files\android_debloat.bat")

The folder accumulated around twenty batch files: launching Genshin Impact, firing up Discord, running ADB commands to strip OEM bloatware from my phone, searching Wikipedia, or setting anime wallpapers.

It was rigid and brittle. If the phrasing changed slightly, the string match missed. But it proved a fundamental principle: language models didn't need to stay trapped in a browser tab. They could be wired directly into local executables.

Eventually, life got busy with studies and other priorities, and the project sat paused on my drive for months. Later on, when modern LLM function calling arrived in model SDKs, I picked it back up. The project evolved into Aiko, rebuilt around direct Win32 integration instead of crude batch files, and was eventually refined and renamed to Kiko as the architecture matured.

3. Under the Hood: Interfacing with Windows Internals

Rather than treating Windows as a black box and relying on high-level GUI automation or simulated mouse clicks, Kiko was built to drop directly down to the operating system's underlying mechanisms. Here is how the core subsystems were actually implemented:

Console Eavesdropping: The IFEO Hijack and a 14-Byte Jump

I wanted Kiko to monitor my terminal in the background and roast me or explain errors whenever a command failed (like git merge conflicts or missing packages), without forcing me to launch a dedicated custom shell wrapper.

My first attempt was creating a custom C++ console process that redirected standard input and output pipes. It failed miserably: standard pipe redirection completely broke interactive terminal features—tab completion died, cursor positioning glitched, and ANSI color sequences refused to render because it bypassed the Windows ConPTY subsystem.

The real hack was dropping directly into the Windows Registry:

By setting a Debugger string value pointing to my custom C++ injector (injector.exe), Windows was tricked into redirecting every single invocation of PowerShell across the entire operating system through my binary.

[Console Eavesdropping Architecture]

User / Terminal Process (powershell.exe)
   │
   ├─► Triggered via IFEO (Image File Execution Options: Debugger)
   ▼
Injector Proxy (injector.exe)
   │  ├─► Spawns powershell.exe with DEBUG_PROCESS (Bypasses recursive IFEO loop)
   │  ├─► Holds child PID & proxies lifecycle (Prevents VS Code node-pty crash)
   │  └─► VirtualAllocEx + CreateRemoteThread -> Injects hook.dll
   ▼
hook.dll (Inside powershell.exe address space)
   │  ├─► Resolves kernelbase.dll!WriteConsoleW
   │  ├─► Overwrites prologue with 14-byte 64-bit indirect jump detour
   │  ├─► Flushes instruction cache: FlushInstructionCache()
   │  └─► Synchronizes stream via CRITICAL_SECTION
   ▼
Console Output Stream
   │
   ├─► Matches "FullyQualifiedErrorId" or stderr signatures
   ├─► Dispatches UDP packet to Kiko daemon (localhost:9999)
   └─► Injects commentary in 24-bit ANSI pink: \x1b[38;2;255;105;180m

That immediately caused two nasty engineering roadblocks:

• The Recursive IFEO Fork Bomb: When injector.exe called CreateProcessW("powershell.exe"), Windows tried to evaluate IFEO again, triggering an infinite recursive loop that froze the machine. The fix was passing DEBUG_PROCESS | DEBUG_ONLY_THIS_PROCESS flags—the Windows NT kernel explicitly skips IFEO evaluation if the process is being spawned under an active debugger.
• The IDE Terminal Crash Mystery: While Windows Terminal worked, launching PowerShell inside VS Code or modern IDEs caused the terminal pane to instantly crash and disappear. Modern IDEs use node-pty, which tracks the exact child PID spawned by CreateProcess. When my injector exited after 5ms, the IDE thought PowerShell died and tore down the pane. The injector had to be rewritten as a blocking, handle-inheriting lifecycle proxy (bInheritHandles = TRUE, WaitForSingleObject).

Once PowerShell spawned cleanly, the injector allocated remote memory (VirtualAllocEx), injected a DLL (hook.dll) via CreateRemoteThread, and hooked WriteConsoleW.

Here lay another trap: on 64-bit Windows, kernel32.dll!WriteConsoleW is merely a 6-byte forwarder stub pointing into kernelbase.dll. Overwriting 14 bytes there corrupts adjacent memory pages and causes instant access violations. The hook had to resolve the address directly inside kernelbase.dll!WriteConsoleW, overwriting the function prologue with an absolute 14-byte 64-bit indirect jump:

Target: kernelbase.dll!WriteConsoleW (64-bit)

[Original Function Prologue]
Address             Bytes                  Disassembly
0x00007FFF89A12010  48 83 EC 38            sub rsp, 38h
0x00007FFF89A12014  48 8B 44 24 60         mov rax, [rsp+60h]
0x00007FFF89A12019  48 89 5C 24 50         mov [rsp+50h], rbx

[After 14-Byte Inline Jump Detour]
Address             Bytes                  Disassembly
0x00007FFF89A12010  48 B8 [ 8-Byte Ptr ]   mov rax, DetourWriteConsoleW
0x00007FFF89A1201A  FF E0                  jmp rax
0x00007FFF89A1201C  90 90                  nop nop  ; 2-byte alignment padding

Because modern console output is heavily multithreaded (background threads spit out ANSI shell integration sequences while the main thread prints commands), the hook wraps calls in a CRITICAL_SECTION and immediately flushes instruction caches across all CPU cores using FlushInstructionCache(GetCurrentProcess(), targetFunc, 14). When an error signature (like FullyQualifiedErrorId) appears in the rolling buffer, the hook pauses execution, sends the error packet over loopback UDP to the assistant, and injects commentary directly into the console in bright 24-bit ANSI pink (\x1b[38;2;255;105;180m) before the user ever sees the next prompt.

Filesystem Search: Reading the Raw NTFS USN Journal in 800ms

A desktop assistant needs to find files instantly. Using Python's standard os.walk() took anywhere from 40 to 90 seconds across multi-terabyte drives because it issues thousands of individual FindFirstFileW / FindNextFileW system calls. Meanwhile, the Windows Search Indexing service was permanently disabled on my LTSC installation to eliminate background disk thrashing and preserve SSD health.

Instead of asking the operating system to traverse directory trees recursively, I bypassed the Win32 filesystem driver entirely:

• Opened raw physical volume handles (\\.\C:, \\.\D:) with administrator privileges (GENERIC_READ | GENERIC_WRITE).
• Issued the low-level I/O control code FSCTL_GET_NTFS_VOLUME_DATA to deduce cluster geometry and verify the filesystem state.
• Streamed Master File Table records directly from disk using FSCTL_ENUM_USN_DATA (NTFS Update Sequence Number Change Journal) in large contiguous memory buffers.
• Converted raw 64-bit File Reference Numbers (FRNs) into absolute filesystem paths using OpenFileById with FILE_FLAG_BACKUP_SEMANTICS and GetFinalPathNameByHandleW.

The C++ scanner splits queries into lowercase tokens and streams through millions of records in memory, locating files across a 1TB NVMe SSD in approximately 800 milliseconds.

Having instantaneous disk search turned into a huge advantage when building higher-level tools later on. For instance, in Kiko's Job Application Orchestrator, if I give her a company email and a job description, she doesn't make me manually browse for a resume. She runs an MFT scan across all drives in milliseconds to locate all resume PDFs, pipes them through a local FAISS RAG sub-agent to select the best match for the role, drafts a tailored cold email, and copies it straight to my Windows clipboard via raw Win32 calls. I will add more tools that take advantage of this later if I get any ideas.

Hardware Telemetry: Querying the Ryzen SMU Through Ring-0

Standard user-mode WMI thermal queries (MSApi_ThermalZoneTemperature) return static dummy values or fail completely on gaming laptop motherboards because thermal diodes route through proprietary vendor Embedded Controller (EC) chips rather than standard ACPI tables.

To monitor my Ryzen 5 7535HS silicon:

• Built a persistent C++ background daemon (cpu_monitor.exe).
• Dynamically loaded AMD's official SDK (Platform.dll) to issue IOCTL control codes to AMDRyzenMasterDriver.sys—a trusted, signed Ring-0 kernel driver.
• The driver speaks directly over the hardware bus with the System Management Unit (SMU) on the silicon, reading internal CPU junction diodes.

[Ring-0 Hardware Telemetry Pipeline]

AMD Ryzen 5 Silicon (Internal SMU Junction Diodes)
       │
       ▼ [Hardware Bus]
AMDRyzenMasterDriver.sys (Signed Ring-0 Kernel Driver)
       │
       ▼ [IOCTL via Platform.dll]
cpu_monitor.exe (C++ Background Daemon)
       │
       ▼ [Dumps junction temperature every 3000ms]
Named Win32 Shared Memory: "Local\KikoRyzenTelemetry" (CreateFileMapping)
       │
       ├─► Python Backend (mmap.mmap reads 4-byte float with 0% CPU overhead)
       └─► Observer Thread (Paints Win32 GDI thermal HUD if Junction > 90°C)

Cold-booting a driver session takes hundreds of milliseconds. To eliminate latency during chat evaluation, the daemon runs continuously, polling every 3000ms and dumping temperatures into a named Win32 shared memory block (CreateFileMapping). Python's mmap module reads the junction temperature with zero CPU overhead. An independent background observer thread continuously monitors this shared memory alongside GPU thermals via nvidia-smi.

I'm honestly pretty particular about my laptop's thermals. My previous Asus laptop had a terrible cooling system, which sent me down a rabbit hole of researching how to keep temperatures down and eventually using ryzenadj. The thermal OCD became permanent: even though my current machine doesn't overheat at all, I still feel like something is missing if the MSI Afterburner overlay isn't active on my screen while gaming, or HWiNFO when doing heavy workloads. So having Kiko monitor both CPU and GPU temps in the background and pop a transparent GDI overlay if thresholds ever breach was a natural addition.

Desktop Perception: Crawling the DWM Z-Order Stack

One of the features I wanted was for Kiko to understand what I was actively coding or looking at without me having to copy-paste code or take manual screenshots. If I asked "why is this function crashing?", she should just know.

The immediate problem with running an assistant from a terminal interface is that calling the standard Win32 GetForegroundWindow() is useless: the foreground window is literally the terminal process executing the assistant itself.

To figure out what application I was actually using right before invoking the prompt, Kiko drops into user32.dll via Python ctypes and walks directly down the Desktop Window Manager (DWM) visual Z-order stack:

# Traversing down the DWM Z-order stack via user32.dll
hwnd = user32.GetForegroundWindow()

# Walk down the visual hierarchy to identify the target application
while hwnd:
    hwnd = user32.GetWindow(hwnd, GW_HWNDNEXT)
    
    # Ignore minimized windows and bare desktop shells
    if user32.IsIconic(hwnd):
        continue
        
    class_name = get_window_class_name(hwnd)
    if class_name in ("Progman", "WorkerW"):
        continue

    # Skip the executing terminal wrapper process
    pid = get_window_pid(hwnd)
    if pid != terminal_pid:
        target_hwnd = hwnd
        break

By skipping minimized windows (IsIconic), ignoring the background desktop shells (Progman, WorkerW), and filtering out the terminal PID, the traversal lands squarely on the topmost visible application underneath.

From there, extracting the window title reveals the context—for instance, VS Code's title exposes the active file path (filename.py - Project - Visual Studio Code). Kiko pairs that title with local workspace paths, reads the raw bytes directly off disk, and loads the active code context straight into the prompt.

GPU-Accelerated Vision: Direct DXGI Framebuffer Capture

Crawling the DWM Z-order stack was great for deducing filenames, but it had a fundamental blind spot: it was completely blind to what was actually being rendered on screen. If I wanted Kiko to evaluate an in-game stat screen, analyze a graphical artifact, or debug a UI layout, inspecting window titles did nothing.

Later on, I developed a native screen reading pipeline that bypasses slow user-space screenshot APIs and reads frames directly from the GPU using the Desktop Duplication API (IDXGIOutputDuplication):

// Duplicating the active display frame straight from GPU VRAM in ~1ms
IDXGIOutputDuplication* pDeskDupl = nullptr;
pOutput1->DuplicateOutput(pD3DDevice, &pDeskDupl);

DXGI_OUTDUPL_FRAME_INFO frameInfo;
IDXGIResource* pDesktopResource = nullptr;
pDeskDupl->AcquireNextFrame(500, &frameInfo, &pDesktopResource);

A native C++ helper (dxgi_capture.exe) duplicates the desktop frame buffer straight out of GPU VRAM in approximately 1 millisecond. Writing uncompressed 4K frames to disk as BMP files would introduce horrible I/O latency and thrash SSD write cycles. Instead, Python pre-allocates a 35MB block of named shared memory directly in the Windows pagefile (mmap.mmap(-1, ...)), and the C++ engine blasts raw BGRA pixel bytes directly into that memory block.

If Multi-Plane Overlay (MPO) or a full-screen exclusive game triggers a black screen, a fallback heuristic immediately invokes GDI BitBlt. The uncompressed frame is then piped straight into the multimodal vision model, giving Kiko true visual perception over my displays in real time.

Background Audio: WinRT System Media Transport Controls

Most virtual assistant tutorials control music through simulated keypresses: synthesizing virtual VK_MEDIA_PLAY_PAUSE keystrokes through keybd_event or SendInput.

Virtual keystrokes are completely blind. They can't report the current track title, they can't tell whether music is currently playing or paused, and they can't target a specific app. If a web browser plays a YouTube video or an ad in the background, it hijacks the global media hook, meaning your simulated keystroke ends up pausing a browser tab instead of your music player.

Windows 10 and 11 route native media sessions through the asynchronous Windows Runtime (WinRT) under Windows.Media.Control. Using the Python winsdk projection, Kiko requests the session manager directly:

from winsdk.windows.media.control import GlobalSystemMediaTransportControlsSessionManager as SMTC

manager = await SMTC.request_async()
sessions = manager.get_sessions()

# Iterate all background sessions instead of trusting the active one
for session in sessions:
    app_id = session.source_app_user_model_id.lower()
    if "spotify" in app_id or "music" in app_id:
        await session.try_toggle_play_pause_operation_async()
        break

Calling manager.get_sessions() allows iterating over all active, background, and suspended sessions. By inspecting the source_app_user_model_id, transport commands (try_toggle_play_pause_operation_async(), try_skip_next_async()) can be routed to the actual media player even when Chrome or Firefox is dominating the foreground audio session.

I integrated the official Spotify Web API later on for catalog searching, custom playlist generation, and hardware device transfers, but I kept this native WinRT SMTC hook in place because it controls any local media player or background audio without needing internet access or API tokens.

Clean Process Spawning: Registry Scraping and Detached Handles

Asking an assistant to launch an application seems simple until you try executing shell commands like start <app>. If an executable isn't registered in the system PATH, Windows triggers an annoying modal GUI error dialog ("Windows cannot find...").

To resolve installed applications dynamically without hardcoding paths or throwing UI errors, Kiko crawls the Windows Registry uninstall hives:

# Registry targets for installed software
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall
HKCU\Software\Microsoft\Windows\CurrentVersion\Uninstall
HKLM\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall

Iterating through these registry subkeys allows reading DisplayName, InstallLocation, and DisplayIcon values. The binary's physical disk path is extracted cleanly, verifying its existence before attempting execution.

There was another subtle Win32 trap: if a Python process spawns a GUI application using standard subprocess.Popen, the child process inherits the parent console's standard input and output handles by default. If the parent terminal closes or expects standard streams to flush, the inherited handles can deadlock or freeze the terminal session.

The fix was passing the Win32 creation flag DETACHED_PROCESS (0x00000008) into the process creation flags:

# Spawning completely detached GUI processes
DETACHED_PROCESS = 0x00000008
subprocess.Popen(
    [resolved_exe_path],
    creationflags=DETACHED_PROCESS,
    close_fds=True
)

This severs handle inheritance completely, allowing games, editors, and tools to spawn independently without locking up the running assistant or tying themselves to the terminal lifecycle.

Bluetooth Radio Control: Forcing Audio Sink Services via Win32

The modern Windows Settings Bluetooth panel is notoriously clunky and slow. Reconnecting paired Bluetooth headphones after stepping away often requires opening Settings, waiting for devices to refresh, and clicking connect multiple times.

I wanted Kiko to connect or disconnect paired devices instantly via voice or chat. The Windows Bluetooth APIs (bluetoothapis.h via Bthprops.lib) require packed memory structures like BLUETOOTH_DEVICE_SEARCH_PARAMS and BLUETOOTH_DEVICE_INFO. Recreating these structures manually in Python via ctypes is brittle and error-prone, so I dropped down to a compact native C++ helper (bt_manager.exe) to let the compiler handle binary struct alignment cleanly.

Enumerating paired hardware was straightforward using BluetoothFindFirstDevice and BluetoothFindNextDevice. But the real challenge was forcing an audio connection without relying on GUI clicks.

The trick was interacting directly with Bluetooth services. Bluetooth audio devices (headphones and speakers) rely on the Advanced Audio Distribution Profile (A2DP), designated by the standard Audio Sink Service Class GUID:

// The Audio Sink Service Class GUID for Bluetooth headphones
// {0000110B-0000-1000-8000-00805F9B34FB}
const GUID GUID_AUDIO_SINK = { 
    0x0000110b, 0x0000, 0x1000, 
    { 0x80, 0x00, 0x00, 0x80, 0x5f, 0x9b, 0x34, 0xfb } 
};

// Forcefully assert the Audio Sink service state on the device
DWORD result = BluetoothSetServiceState(
    hRadio, 
    &device_info, 
    &GUID_AUDIO_SINK, 
    BLUETOOTH_SERVICE_ENABLE
);

Calling BluetoothSetServiceState with BLUETOOTH_SERVICE_ENABLE forces the Windows Bluetooth kernel stack to immediately assert the A2DP profile on the target device, establishing the audio link instantly. Passing BLUETOOTH_SERVICE_DISABLE tears down the link on demand—bypassing the Windows Settings app entirely.

Dynamic Privacy: Bypassing the camsvc ConsentStore

When checking local weather conditions, Kiko needs geolocation coordinates. If Windows Location permissions are disabled in system settings, an assistant usually just errors out or tells you to manually open Settings. Similarly, I wanted a global privacy killswitch: being able to tell Kiko "turn off my microphone and webcam right now" and having the operating system enforce it immediately.

Official Windows Runtime permission APIs (AppCapability::RequestAccessAsync) only allow a process to request access for itself. Microsoft does not provide a public API for third-party software to globally flip master privacy toggles for the entire operating system.

Looking into Windows internals, privacy capabilities (Location, Webcam, Microphone) are enforced at the service level by the Capability Access Manager Service (camsvc). And camsvc reads its policies directly from the ConsentStore registry keys:

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\<capability>\Value
HKCU\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\<capability>\Value

I could have written a C++ binary to alter these keys, but that would have just been a redundant wrapper around RegOpenKeyEx and RegSetValueEx. Python's built-in winreg module hits the exact same configuration store natively with zero compilation overhead:

import winreg

def set_master_capability(capability: str, allow: bool):
    value_str = "Allow" if allow else "Deny"
    path = rf"Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\{capability}"
    
    with winreg.OpenKey(winreg.HKEY_CURRENT_USER, path, 0, winreg.KEY_SET_VALUE) as key:
        winreg.SetValueEx(key, "Value", 0, winreg.REG_SZ, value_str)

By flipping the Value string between "Allow" and "Deny", Kiko can temporarily unlock Windows Location to grab real-time coordinates for weather forecasts and immediately lock the capability back down, or act as an instant master killswitch for hardware sensors.

4. The Dead Ends: Memory Ripping and the 4GB VRAM Wall

Genuine tinkering inevitably hits brick walls, and documenting what failed is just as valuable as what worked:

5. Looking Back: Computing as a Personal Playground

At first glance, an "AI desktop assistant" sounds like the most generic project imaginable: an API key, a basic prompt, and calling it a day like every other tutorial on the internet.

In reality, software is defined by its mechanisms, not SaaS hype or college viva grading rubrics. Building Kiko wasn't about entering hackathons, chasing venture capital, or making a corporate productivity tool. It was about taking an emerging technology and using it in my own way—as an excuse to explore assembly detours, kernel driver IOCTLs, raw NTFS structures, and Windows subsystems that are usually hidden behind layers of abstraction.

Personal computing has always been about curiosity and tinkering. Sometimes, the most rewarding project you can build is simply one made for yourself, on your own hardware, just to see how the system actually works.