Getting Started with Local AI
Embarking on regular AI usage while hesitant to share personal data with cloud services makes running powerful models locally an appealing alternative. Testing whether local AI is genuinely useful on high-end hardware helps clarify if the investment is worthwhile.
Learning to use local AI is exciting, overwhelming, and frustrating
It is like using cloud-based AI tools, but the privacy makes it more appealing.
Baby's first local AI agent.
Learning to use local AI is exciting, overwhelming, and frustrating
It is like using cloud-based AI tools, but the privacy makes it more appealing.
Apple and Windows desktop hardware are increasingly marketed for local AI and agentic workflows. Running a local assistant that answers only to the user rather than corporate cloud services provides a different experience than relying on subscription chatbots. With data privacy maintained locally, users can find specialized busywork to delegate.
Testing local large language models across multiple Mac and Windows systems provides insight into practical capabilities and setup challenges.
1/4
Hermes Agent can be used with cloud or local models.
Setting Up Self-Hosted Agents
The open-source Hermes Agent desktop application supports macOS, Windows, and Linux, and remains free to use with local large language models. Systems with substantial unified memory can handle massive models with billions of parameters. Because local execution avoids per-token costs, users can experiment with very large models.
Using straightforward onboarding interfaces allows users to select specific models, such as the 125-billion parameter Qwen model. Testing smaller models across various laptops and desktop processors also offers a view of hardware scaling.
Getting the software running and integrated with chat applications like Telegram is relatively straightforward, though figuring out practical use cases remains the main hurdle.
What do I do with this thing?
My Hermes daily briefing, sent to me daily via Telegram. Basic, I know.
Practical Automation and Limitations
Initial automation experiments can include small tasks like generating a daily morning briefing via a scheduled cron job that scans email, calendar entries, and weather reports. Ensuring the host computer remains awake is necessary for scheduled background tasks to execute properly.
A more substantial task involves reorganizing large game libraries. Instead of manual sorting, a local AI agent can review an installed Steam client and categorize hundreds of titles by genre while preserving custom user categories.
Granting local AI tools permissions via web API keys enables them to execute organizational tasks locally. These API keys can be revoked immediately after the heavy lifting is complete.
1/2
Starting the process of having Hermes organize my Steam library.
Data Analysis and Benchmarking
Additional local tasks include data analysis of sensitive financial records and creating spec comparison spreadsheets for unreleased products under embargo. Keeping this information on a local machine prevents privacy risks associated with cloud uploads.
More complex automation projects, such as writing scripts to run laptop benchmark tests, require iterative prompting and debugging. Developing these workflows remains a work in progress.
Apple and other hardware makers demonstrate multi-device compute clusters for local AI, highlighting the high hardware requirements for intensive local workloads.
Conclusions on Local AI
While powerful local models offer privacy benefits, they are not magical assistants capable of flawless autonomous execution, and automated tasks can still break unexpectedly.
Treating local AI strictly as a computer program rather than a conversational companion helps maintain a pragmatic perspective on its operational utility.
Continued testing and caution are essential for understanding the actual value and limitations of local artificial intelligence tools.
