Technical Research & Engineering

GLOBAL INTENT COMPANY

Interactive technical environment for machine learning, scientific instrumentation, computational systems, experiments, and project evidence.

Developed independently by founder and researcher Cory Tortorici. Source code, controlled experiments, null results, architectural iterations, and public research logs are made inspectable across all project nodes.

Source Repositories

Laboratory Chronology

LATEST FROM THE LAB

Recent empirical findings, architectural iterations, and null results surfaced directly from active research logs.

Active laboratory research log entries with timestamped records are available in the full research log.

Interactive Spatial Knowledge Graph

EXPLORE THE RESEARCH

Navigate the connected network of research projects, architectural pipelines, and domain clusters in real-time 3D.

Rotate • Zoom • Click Nodes to Unfold

60 FPS

Tier:

Focused on TinyCoherent • Orbiting sub-nodes active

Domains:

SPATIAL FOCUS:TinyCoherent•Machine Learning

GitHub

Machine LearningKind: researchActive Research / Pure C LLM Engine[TINYCOHERENT-SCALE]

TinyCoherent

GI-Company/tinyCOHERENT

An experimental pure C Transformer training and inference engine built for Apple Silicon. Explores 3.45M and 23M parameter architectures, subword BPE tokenization designed to address character-level sequence expansion, pure C conversational instruction fine-tuning with prompt-loss masking, pre-allocated activation buffer inspection via a live terminal REPL, and the Rung 6 V2 reasoning dataset curation workflow.

Core Research Question:"Can an inspectable glass-box Transformer language model be trained, instruction-tuned, and scaled entirely in pure C directly on Apple Silicon without dependency on Python/PyTorch runtimes, while maintaining rigorous reasoning token curation?"

Engineering Objective:Build a zero-dependency C deep learning engine that executes training and inference with custom memory-mapped tensor buffers, Accelerate BLAS linear algebra, and internal layer observability via a live terminal REPL.

Structural Overview

TinyCoherent implements an autoregressive decoder-only Transformer entirely in ANSI C with zero third-party framework dependencies. The system architecture coordinates contiguous memory-mapped tensor buffers, custom analytical reverse-mode backpropagation, cache-aligned matrix multiplication routines via Apple Accelerate BLAS, causal self-attention, and an interactive glass-box REPL for inspecting internal activations in real-time.

Canonical Source Artifact

GitHub Repository

Primary Source CodePublic

GI-Company /tinyCOHERENT

Canonical public source repository: Pure C LLM training engine with Apple Silicon hardware acceleration, Byte-Pair Encoding subword tokenization, glass-box activation inspection, instruction tuning, attribution tests, scaling ledgers, and curated reasoning scratchpad datasets.

Canonical GI-Company Source

VIEW SOURCE

Design Invariants & Principles

[1]Zero Framework Overhead: Standalone ANSI C implementation without Python, PyTorch, CUDA, or external math framework dependencies (SOURCE-VERIFIED).

[2]Contiguous Activation Buffer Inspection: Layer activations and attention weights reside in pre-allocated memory buffers, enabling constant-time O(1) buffer lookup during generation without dynamic heap allocation per inspection tap (DESIGN CLAIM / HYPOTHESIS).

[3]Data Curation Integrity: Prioritizes verified step-by-step scratchpad examples over raw synthetic token volume, discarding unverified candidate tokens (AUTHOR-REPORTED).

[4]Hardware-Targeted Implementation: Integrates Apple Accelerate BLAS (cblas_sgemm) on unified memory architectures (SOURCE-VERIFIED; compute bandwidth bounds AUTHOR-REPORTED).

Sequential Pipeline & Processing Stages

Token Ingestion & EmbeddingSequence Encoding

Subword BPE lookup mapping token IDs to dense embedding vectors with learned positional encodings.

Layer-by-Layer Causal Transformer BlocksContextual Self-Attention

LayerNorm / RMSNorm, multi-head causal self-attention with KV caching, and feed-forward MLP blocks in pure C.

Glass-Box Activation TapReal-Time Introspection

Direct reading of pre-allocated activation buffers allowing the live REPL to query attention distributions and intermediate layer activations without dynamic allocation.

Loss Masking & Autograd BackpropagationTraining Optimization

Computes analytical cross-entropy gradients over response tokens only, executing backprop directly into parameter gradient buffers.

Technology Stack

ANSI C (C99/C11 Standard)Apple Accelerate Framework (cblas_sgemm / BLAS)POSIX Memory Mapping (mmap / msync)Custom Byte-Pair Encoding (BPE) Subword TokenizerPure C AdamW Optimizer with Decoupled Weight Decay

Technical Decisions

Contiguous single-buffer memory allocation per forward-backward pass to eliminate dynamic malloc fragmentation during training loops.

Prompt-loss masking in pure C during conversational fine-tuning: gradients are computed exclusively on target response tokens.

Subword Byte-Pair Encoding vocabulary trained on domain corpora to mitigate sequence length inflation associated with character-level tokenization.