Distributed Graph Processing in Practice: From Python Parallelism to Rust and Go

Introduction

Large-scale graph processing is a recurring challenge in both security analytics and systems research. In my recent work on dependency graphs and attack-chain analysis, I experimented with multiple parallel and distributed execution models to scale graph computation efficiently.

This post reflects on a comparative exploration of several approaches: earlier Python-based solutions using Numba and Joblib, and more recent systems-level implementations in Rust and Go. Rather than focusing on benchmarks alone, I discuss the trade-offs I encountered in terms of performance, memory behavior, concurrency control, and engineering complexity.

The goal of this post is to provide a practitioner-oriented perspective on how different language and runtime choices affect the design of scalable graph processing pipelines in real-world systems.