DJ action (the memories!)

Running your own AI LLM locally on your computer

 

I recently purchased a new M4 Macbook Pro and was keen to see how well it performs. What better way then trying to run a local AI language model. While there are different ways to deploy LLMs, this post focuses on the quick and easy way and aims to compare it to what is available in the cloud. Why do you want to run a local LLM you may ask? Firstly the fact that it can be run offline and all of your prompt details are not sent to a third party (i.e. OpenAI / meta, google, etc.) and secondly because you can :)

For this post I am using LMStudio (https://lmstudio.ai/) which is extremely simple to download and install (uses approx 500MB of space). Once installed you are prompted to choose a model with the default being OpenAI's gpt-oss-20b (~12GB) - which you can read more about here.

Once the model is downloaded you can "start a new chat". Below is my idle system usage prior to launching the model

And when the model has loaded

And when its in use. As you can see memory doesnt overly change but CPU usage increases when the prompt is returning. Further investigation into this shows that the GPU is used when the prompt is returning.

After reading some posts around speed and response i was reasonablly surprised it was quite performant taking less than a couple of seconds to respond. LMStudio also supports RAG so i may look to update this post with more details of RAG and or performance comparison with other LLMs which can be loaded into LMStudio.

Some considerations are to increase the token count when loading from the default 4096 tokens

Running Linux on Microsoft Surface 2 RT

The Microsoft Surface 2 is an ARM32 based tablet which is largely defunct for many reasons which you can read all about online. The aim was to run Linux on this device to enable better application support and a modern web browser.

Journey 

The following openrt link was super helpful in understanding how feasible it is running Linux on the Microsoft Surface 2 instead of Windows RT 8.1 and how to go about jail breaking the secure boot. All was going well until when trying to boot linux from a USB drive where the EFI boot prompt would just hang for several minutes and not load, then continue and error out.

Initially I thought I had a Surface 1 tablet and wasnt aware it was a Surface 2 until i tried to access the UART serial port to do further debugging. The give away was when trying to disassemble that there are no screws as shown below (Surface 1) under the kickstand. Also I was probably using the wrong device tree file (dtb) which wouldn't help.

Knowing I had a Surface 2, limited details on the Open Surface RT website and the Discord server being down I started to think the hardware support just wasnt there. After reading this guide (and alot of trial and error) it seemed like the best or quickest path to success would be trying to load an EXT file system on the internal MMC by shrinking the Windows 8 RT file system (mmcblk3) and creating additional partitions with the free space. 

There is a bunch of items disabled in the tegra114-microsoft-surface-2.dtb which is getting around which likely due to people trying to get the surface2 working. You can download an updated dtb file here which enables the SD card and enables all disabled devices (though nothing else useful seems to work when enabling all)

Things that dont seem to work

At the time of writing and as far as i'm aware there seemed to be issues with USB support on the surface 2 in linux (wifi and SD works fine) which means it cant be used or can be used under certain cirmstances (e.g. USB boot and copy into RAM). It could be just a linux USB driver issue but haven't had time to look into it further. Sound seems to work but havent tested the microphone and the webcam doesnt seem to work.

Another thing worth a mention is the power and battery details don't appear to be working from the desktop. You can run upower to figure out if the device is plugged in or not, but can't seem to get the battery charge details displaying. Also when you shutdown the machine it doesnt seem to power off the device (screen just goes black) which means its still drawing power. Guessing ACPI isnt working as needed.

Method

The following assumes you have a jailbroken Surface 2 tablet with secure boot disabled, golden keys installed and a few USB keys (=>8gb). This method with the dtb file update could likely be modified to deploy to the SD card (mmcblk1p1) rather than internal MMC (mmcblk0p5)

1. Download the following files from open-rt.party and copy to a USB drive
1. https://files.open-rt.party/Linux/Other/surface-2-linux-resizepart-emmc.zip
2. https://github.com/andrewjameslee/surface2/blob/main/tegra114-microsoft-surface-2-all.dtb
2. Download Raspberian - 2022-09-22-raspios-bullseye-armhf
1. Theres a bunch of work needed for this:
1. pull out root partition from image using dd and partx
2. fix /etc/fstab for /dev/mmcblk0p5 to be the / partition
3. put kernel modules to the /lib/modules/ directory to get the wifi working
3. Boot into Windows RT and load disk management
4. Shrink the windows partition to the smallest size (~16GB) and delete the recovery partition
5. Create two new FAT32 paritions 
1. Partition 5 - FAT32 - ~6 GB
2. Partition 6 - FAT 32 - ~5GB
6. Copy the raspios-bullseye-root-image onto partition 6 (~4GB) and potentially kernel modules (used in later steps)
7. Create Linux Bootable USB (Raspberian - 2022-09-22-raspios-bullseye-armhf using pi imager then copy boot files from surface-2-linux-resizepart-emmc.zip)
8. Ensure the commandline.txt is configured correctly on USB (initrd=initrd.gz, root=/dev/ram0, init=/init.sh)
9. Boot into Linux from USB
1. mkdir /a
2. mount /dev/mmcblk0p6 /a
3. dd if=/a/raspios_armhf-2022-09-26/2022-09-22-raspios-bullseye-armhf-root.img of=/dev/mmcblk0p5 bs=1M status=progress
4. mkdir /b
5. mount /dev/mmcblk0p5 /b
6. cp -Rp /a/lib/modules/* /b/lib/modules/
7. nano /etc/fstab (add /dev/mmcblk0p5 / and hash out other entries)
8. umount /a
9. unmount /b
10. sync
11. CTRL-D (reboot)
12. Ensure the commandline.txt is configured correctly on USB (root=/dev/mmcblk0p5) and a similar screen to the following should pop up

Once you've configured the setup you should get the usual desktop like the below

Confirmation the device is indeed a Surface 2

Boot from internal flash instead of USB

There is good background detail on the UEFI Boot Sequence on Open Surface RT. The following steps describe how to get Linux booting from the internal mmc drive without the need to boot from USB.

• mount EFI partition (linux = mmcblk0p2 | windows disk 0 partition 1)
• copy boot files to EFI partition (microsoft/boot, test.efi (shim to boot.efi), boot.efi (linux kernel), tegra114-microsoft-surface-2.dtb, commandline.txt, config.txt
• rename microsoft/boot to microsoft/boot_win
• copy in microsoft/boot to microsoft/boot which will load the test.efi policy to enable linux to boot
• can edit boot sequence from EFI partion in linux (mount /dev/mmcblk0p2)
• can edit boot sequence from EFI partion in windows (/dev/mmcblk0p2) diskpart, list disk, select disk 0, list partition, select partition 1, assign letter=b, exit

These steps allow for you to automatically reboot into your OS of choosing though it doesnt give you a nice menu selector with a time out like say GRUB does. Further consideration is to look into bcdedit to add a second boot menu option for test.efi though need to consider if you reboot the surface and you are prompted to choose a boot option this may not be possible if you are accessing remotely.

Other considerations 

Other options considered which could be of interest to people:

• consider compiling ARM32 ext driver for windows 8 - seems like cygwin would need to be compiled first, no one has compiled to date based on list of arm windows software
• other distributions like ubuntu 18 server seems to run on the surface2
• cross compiling custom kernels

• #apt install git make gcc g++ device-tree-compiler bc bison flex libssl-dev libncurses-dev python3-ply python3-git libgmp3-dev libmpc-dev gcc-arm-linux-gnueabihf

• #export ARCH=arm

  #export CROSS_COMPILE=/usr/bin/arm-linux-gnueabihf-

• #make surface-2_defconfig

• Confirm the following is in .config

  CONFIG_EFI=y

  CONFIG_EFI_STUB=y

• #make

• Compiled zImage will be in arch/arm/boot/zImage which can be renamed to boot.efi

• initramfs attached to zImage (EFI Stub)
• compile kernel with CONFIG_INITRAMFS_SOURCE="/initram/fs/root/directory"

The Cosmic Symphony: Where Faith and Science Dance

In the grand orchestra of existence, religion and science often play separate melodies, their harmonies rarely intermingling. Yet, beneath the surface, a deeper rhythm pulsates, one that binds the human spirit to the vast cosmos. It's a rhythm that whispers of prophets and particles, of angels and algorithms, of the divine woven into the very fabric of reality.

In Christianity, the trinity serves as a potent metaphor for this cosmic dance. The Father, embodied in the prophets and societies of men, represents our earthly stage, our collective striving towards a higher purpose. The Son, within each individual, is the spark of divinity, the unique voice whispering truths from within. And the Holy Spirit, encompassing the universe, is the silent symphony itself, the boundless mystery that science endeavors to unravel.

Islam echoes this sentiment with its unwavering emphasis on the will of Allah, a will that permeates the universe, society, and even our own choices. It's a will that grants freedom and responsibility in equal measure, a constant dialogue between the individual and the infinite.

But these melodies, as beautiful as they may be, can be corrupted. Religion, for all its power to elevate, can be twisted by human hands, its teachings weaponized for power and control. This, perhaps, is the seed of atheism, a rebellion against the misuse of faith, a yearning for uncorrupted truth.

Science, too, has its own limitations. While it illuminates the physical world with breathtaking clarity, it often leaves the human spirit yearning for something more. The cold equations and sterile labs can feel devoid of the warmth and wonder that faith provides.

Yet, it's in this very tension, this friction between the spiritual and the scientific, that the true potential of both lies. Science, with its relentless pursuit of understanding, can become a tool to decipher the divine script written in the stars and etched in the human genome. Religion, with its focus on the human heart, can infuse scientific advancements with compassion and purpose, ensuring they serve the betterment of all.

Imagine a world where medical advancements are not just technical marvels but acts of love, guided by the desire to heal not just bodies but souls. Imagine a society where the teachings of prophets are not used to divide but to unite, reminding us of our shared place within the cosmic symphony.

This is not a call for blind faith or unquestioning obedience. It is a call to listen, to resonate with the melodies of both religion and science, to find the harmonies that bind them. For it is in this confluence, in the dance between the finite and the infinite, that we may glimpse the true nature of our existence, the divine spark within each atom, the cosmic echo in every heartbeat.

So let us raise our voices, not in divisive chants, but in a unified hymn, a chorus of faith and reason, of spirit and matter. Let us dance to the rhythm of the universe, a dance where science illuminates the path and religion guides the heart, a dance where the seeker finds solace in the song of the cosmos. For in the end, it is not about choosing between religion or science, but about embracing the symphony they create together, a symphony that whispers the secrets of existence to all who have ears to hear.

Disclaimer: Google's Artifical Intelligence has been used to generate this blog using all the information available to it in its model.

Hypothesis for Further Investigation: Quantum Entanglement for Revolutionary Memory Storage

Central Idea: Exploiting the unique properties of entangled particles to create a revolutionary memory storage system that transcends the limitations of traditional methods.

Rationale:

Current limitations: Traditional memory technologies like RAM and SSDs face fundamental trade-offs between performance (speed) and redundancy (data persistence). RAM offers high speed but volatile storage, while SSDs provide persistent storage but with slower access times.

Quantum advantage: Quantum entanglement allows for the creation of linked particles whose fates are intertwined, regardless of their physical separation. This phenomenon offers unique possibilities for data storage:

Simultaneous storage: Entangled particles can be used to encode information in both memory and disk simultaneously, enabling instant access to data stored on disk.

Quantum error correction: Entanglement can be leveraged to detect and correct errors in stored data, enhancing data integrity and redundancy.

Fault tolerance: Entangled particles can be distributed across multiple physical locations, making the system resilient to hardware failures and data loss.

Proposed investigation:

Develop protocols for encoding and decoding information in entangled particles. This involves exploring various qubit states and entanglement configurations to optimize data representation and manipulation.

Investigate methods for integrating entangled particle storage with existing memory and disk architectures. This includes designing interfaces and protocols for seamless data transfer and access.

Simulate and model the performance and reliability of the proposed system. This will involve analyzing factors like error rates, data throughput, and scalability to assess the feasibility and potential impact of the technology.

Explore potential applications of this technology beyond traditional data storage. Examples include quantum cryptography, secure communication, and distributed computing.

Expected outcomes:

A proof-of-concept for a novel memory storage system that combines the speed of RAM with the persistence of disk storage.

Enhanced data integrity and redundancy through quantum error correction and fault tolerance.

Foundations for further research and development in quantum computing and information processing.

Challenges and considerations:

Maintaining the coherence of entangled particles in real-world environments is a significant challenge.

Integrating quantum systems with existing infrastructure requires careful design and engineering.

The cost and complexity of implementing quantum memory technology may initially limit its adoption.

Conclusion:

Quantum entanglement offers a promising avenue for revolutionizing data storage by combining performance and redundancy in a way that traditional technologies cannot. Further investigation into this area has the potential to significantly impact computing, communication, and other fields.

This hypothesis is just a starting point for further research and development. As the field of quantum computing continues to evolve, we can expect even more innovative approaches to data storage and information processing to emerge.

Disclaimer: Google's Artifical Intelligence has been used to generate this paper using all the information available to it in its model.

Hypothesis for Further Investigation: Using the Wilkinson Microwave Anisotropy Probe (WMAP) for Space Navigation around Black Holes and Stars

Abstract: This hypothesis proposes that the Wilkinson Microwave Anisotropy Probe (WMAP) can be repurposed as a novel space navigation system by leveraging its unprecedented sensitivity to the Cosmic Microwave Background (CMB) radiation. This could provide a unique and potentially superior method for navigating around black holes and stars compared to current systems relying on radio or optical signals.

Rationale:

CMB as a Reference Frame: The CMB is a uniform, isotropic radiation field permeating the entire universe. Its precise temperature and polarization fluctuations serve as a highly stable reference point in any location.

WMAP's Superiority: WMAP boasts unmatched sensitivity and angular resolution for CMB measurements, enabling it to detect even the subtlest variations in the radiation field.

Black Hole and Star Signatures: The gravitational influence of black holes and stars distorts the CMB locally, creating unique signatures in its temperature and polarization patterns. WMAP, with its exceptional precision, could detect these signatures and use them to determine the relative position and mass of these celestial objects.

Navigation Potential:

Black Hole Detection and Avoidance: By measuring CMB distortions, WMAP could identify the presence of black holes and map their event horizons, allowing spacecraft to safely navigate around them.

Star System Mapping: WMAP could create a detailed map of the CMB distortions surrounding stars, revealing their mass distribution, planetary systems, and potentially habitable zones. This information could be crucial for interstellar travel and the search for exoplanets.

Interstellar Positional Awareness: By comparing local CMB measurements with a pre-recorded map of the CMB, WMAP could determine the relative position and orientation of a spacecraft within the universe, independent of traditional radio or optical navigation methods.

Challenges and Further Investigation:

Signal Processing: Developing algorithms to accurately extract black hole and star signatures from the CMB data requires advanced signal processing techniques.

Calibration and Error Correction: Precise calibration of WMAP and accounting for instrumental errors are crucial for reliable navigation.

Testing and Validation: Extensive simulations and ground-based experiments are needed to validate the feasibility of using WMAP for space navigation before in-flight testing.

Potential Benefits:

Enhanced Safety: Accurately detecting and avoiding black holes would significantly improve the safety of interstellar travel.

Improved Efficiency: Efficient navigation around stars and within star systems could drastically reduce travel times.

Independent Navigation: Reliance on CMB, a ubiquitous and unchanging reference frame, eliminates dependence on external communication infrastructure.

Conclusion: This hypothesis proposes a novel application for WMAP as a space navigation system around black holes and stars. While challenges remain, the potential benefits for interstellar travel and exploration are immense. Further research and development are warranted to test and refine this concept, potentially revolutionizing our approach to navigating the vast expanse of space.

Note: This hypothesis is speculative and requires further research and development before it can be implemented as a practical navigation system. However, it presents a promising and theoretically sound direction for future space exploration technology.

Disclaimer: Google's Artifical Intelligence has been used to generate this paper using all the information available to it in its model.