Benchmark Note: Noodle vs TensorFlow Lite Micro for Image Denoising on ESP32-S3
1. Benchmark objective
In this benchmark, we compare the proposed Noodle runtime against TensorFlow Lite Micro (TFLite Micro) for the same image denoising autoencoder deployed on an ESP32-S3 board. Our objective is to compare the runtime behavior of two embedded inference approaches under the same input/output protocol:
- Noodle, which executes the network using explicit operator calls and grow-only
NoodleBuffertensor buffers. - TFLite Micro, which executes the same model through a general interpreter and a preallocated tensor arena.
We focus on inference latency, reconstruction quality, and runtime memory behavior.
2. Target platform
We executed the benchmark on an ESP32-S3 class board using the Arduino framework with PSRAM enabled. We transferred images from the host computer to the microcontroller over serial, and the denoised output image was returned to the host using the same serial protocol for both runtimes.
We report measurements based on 100 test images. We measured runtime memory before and after inference to check whether inference caused changing free memory or memory leakage.
3. Model under test
We evaluate a small convolutional denoising autoencoder for 28 × 28 grayscale images. The model receives a noisy image and reconstructs a denoised output image of the same spatial size.
We use the following network structure:
| Stage | Operation | Channels | Kernel | Stride | Padding | Activation | Output size |
|---|---|---|---|---|---|---|---|
| Input | Image | 1 | - | - | - | - | 28 × 28 × 1 |
| Encoder 1 | Conv2D | 1 → 16 | 3 × 3 | 2 | same | ReLU | 14 × 14 × 16 |
| Encoder 2 | Conv2D | 16 → 32 | 3 × 3 | 2 | same | ReLU | 7 × 7 × 32 |
| Decoder 1 | Conv2DTranspose | 32 → 16 | 3 × 3 | 2 | same | ReLU | 14 × 14 × 16 |
| Decoder 2 | Conv2DTranspose | 16 → 1 | 3 × 3 | 2 | same | Sigmoid | 28 × 28 × 1 |
In the Noodle implementation, we store tensors in channel-first format during inference. We execute the network using two grow-only NoodleBuffer objects in a ping-pong pattern. Internal operator scratch buffers are managed automatically by Noodle.
4. Runtime implementations
4.1 Noodle implementation
Our current Noodle implementation uses NoodleBuffer, a grow-only tensor buffer abstraction. A NoodleBuffer starts empty, grows when a layer requires more storage, and then reuses the same memory for later layers. It does not shrink automatically during inference. Thus, we define two tensor buffers:
static NoodleBuffer A;
static NoodleBuffer B;
We write the forward pass as an explicit ping-pong schedule:
NoodleBuffer *in = &A; // X
NoodleBuffer *out = &B;
W = noodle_conv_float(in, 1, 16, out, W, E1, no_pool, NULL); // B = Z1
swap_buffers(in, out);
W = noodle_conv_float(in, 16, 32, out, W, E2, no_pool, NULL); // A = Z2
swap_buffers(in, out);
W = noodle_conv_transpose_float(in, 32, 16, out, W, D1, NULL); // B = Z3
swap_buffers(in, out);
W = noodle_conv_transpose_float(in, 16, 1, out, W, D2, NULL); // A = Y
swap_buffers(in, out);
noodle_sigmoid(in, IMG_SIZE);
The largest individual tensor in the model is:
16 × 14 × 14 = 3136 floats
However, because NoodleBuffer grows only when a specific buffer needs more capacity, the two ping-pong buffers do not necessarily grow to the same size. In our measured steady-state run, the capacities were:
A_cap = 1568 floats
B_cap = 3136 floats
This matches the actual tensor flow:
A contains X = 784 floats
B contains Z1 = 3136 floats
A contains Z2 = 1568 floats
B contains Z3 = 3136 floats
A contains Y = 784 floats
Thus, the visible steady-state tensor-buffer storage is:
A = 1568 × 4 bytes = 6,272 bytes
B = 3136 × 4 bytes = 12,544 bytes
A + B = 18,816 bytes ≈ 18.4 KiB
Internal scratch storage is managed automatically by Noodle. The measured PSRAM usage reported below therefore includes the grown NoodleBuffer tensors, internal scratch allocation, allocator overhead, and runtime/library bookkeeping.
4.2 TensorFlow Lite Micro implementation
For TensorFlow Lite Micro, we use a .tflite model converted from the same autoencoder. We execute the model using a MicroInterpreter and a preallocated tensor arena.
The model size was:
42,668 bytes ≈ 41.7 KiB
After tuning, the model successfully allocated and executed with a tensor arena of:
32 KiB
We used AllOpsResolver to avoid missing-operator errors during integration. This provides generality but may increase flash and runtime overhead compared with a minimal operator resolver.
5. Measurement method
For each runtime, we transmitted 100 noisy test images to the ESP32-S3. For each image, the microcontroller returned:
- the inference time in microseconds,
- the denoised output image,
- runtime memory measurements.
We then computed:
- mean noisy MSE,
- mean output MSE,
- MSE reduction,
- mean inference time.
We measured runtime memory before and after inference. The memory measurements show constant values before and after inference for both runtimes. Therefore, both implementations exhibit fixed runtime memory behavior after setup or first buffer growth.
6. Benchmark results
6.1 Latency and reconstruction quality
| Metric | Noodle | TFLite Micro | Observation |
|---|---|---|---|
| Mean inference time | 103,765.1 µs | 194,361.6 µs | Noodle is faster |
| Mean inference time | 103.77 ms | 194.36 ms | Noodle saves 90.60 ms per image |
| Throughput | 9.64 images/s | 5.14 images/s | Noodle has higher throughput |
| Mean output MSE | 0.003889 | 0.004051 | Comparable reconstruction quality |
| Mean MSE reduction | 88.15% | 87.66% | Comparable denoising performance |
The Noodle implementation is approximately:
194.36 / 103.77 ≈ 1.87× faster
than the TensorFlow Lite Micro implementation.
6.2 Memory usage
The measured steady-state PSRAM free values are:
| Runtime | PSRAM free | Estimated PSRAM used |
|---|---|---|
| NoodleBuffer Noodle | 16,755,364 B | 21,852 B |
| TFLite Micro, 32 KiB arena | 16,740,776 B | 36,440 B |
Assuming a total PSRAM size of 16,777,216 bytes, the estimated PSRAM usage is:
Noodle PSRAM used = 16,777,216 - 16,755,364
= 21,852 bytes
≈ 21.3 KiB
TFLite Micro PSRAM used = 16,777,216 - 16,740,776
= 36,440 bytes
≈ 35.6 KiB
Thus, with the tensor arena reduced to 32 KiB, TensorFlow Lite Micro used approximately:
36,440 - 21,852 = 14,588 bytes ≈ 14.2 KiB
more PSRAM than Noodle in this benchmark.
| Metric | Noodle | TFLite Micro | Meaning |
|---|---|---|---|
| Free PSRAM after setup/growth | 16,755,364 B | 16,740,776 B | Higher is better |
| Estimated PSRAM used | 21,852 B | 36,440 B | Lower is better |
| Difference | - | 14,588 B more than Noodle | TFLite Micro uses about 14.2 KiB more |
| Runtime allocation behavior | Fixed after buffer growth | Fixed after setup | No per-image memory growth |
The measured steady-state NoodleBuffer capacities were:
| Buffer | Capacity | Bytes |
|---|---|---|
A |
1568 floats | 6,272 B |
B |
3136 floats | 12,544 B |
A + B |
4704 floats | 18,816 B |
The important observation is that buffer A did not grow to the global maximum tensor size. It only grew to the largest tensor that actually occupied A during the ping-pong schedule.
7. Interpretation
Both runtimes show fixed memory usage during repeated inference. The free memory before inference and after inference remained unchanged, indicating that neither implementation performs per-image dynamic allocation or leaks memory during the measured inference loop.
Noodle uses less PSRAM and achieved substantially lower inference latency. The main difference is the execution strategy. Noodle uses direct operator calls and a model-specific execution schedule expressed through NoodleBuffer ping-pong tensors. TensorFlow Lite Micro uses a general interpreter and tensor arena, which provide broader model compatibility but introduce interpreter overhead.
The newer NoodleBuffer interface also improves usability. Earlier examples required manual tensor-buffer and scratch-buffer allocation. With NoodleBuffer, the user still sees the embedded execution pattern, but the buffers grow automatically when needed and are reused afterward. This keeps the ping-pong memory model explicit while making examples easier to read and teach.
For this autoencoder workload, we achieved comparable reconstruction quality with Noodle while reducing inference time by approximately 1.87× and using less PSRAM than a tuned TensorFlow Lite Micro configuration.
8. Summary
For the same 28 × 28 image denoising autoencoder on ESP32-S3, Noodle achieved a mean inference time of 103.77 ms, compared with 194.36 ms for TensorFlow Lite Micro. This corresponds to a speedup of approximately 1.87×. Both runtimes achieved comparable reconstruction quality, with mean output MSE values of 0.003889 for Noodle and 0.004051 for TensorFlow Lite Micro. Memory measurements showed fixed runtime behavior for both implementations, with no change in free memory before and after inference. With a tuned 32 KiB tensor arena, TensorFlow Lite Micro used approximately 35.6 KiB of PSRAM, while Noodle used approximately 21.3 KiB. These results indicate that Noodle's direct operator execution and explicit ping-pong memory schedule reduce latency while maintaining a slightly smaller memory footprint than a tuned interpreter-based runtime.
The comparison with TensorFlow Lite Micro highlights the trade-off between a general interpreter-based deployment path and an explicitly scheduled runtime. TensorFlow Lite Micro provides broad model compatibility through an interpreter and tensor arena, but this generality introduces additional execution overhead. In contrast, Noodle uses a model-specific execution schedule with grow-only NoodleBuffer tensors, allowing the inference path to avoid per-image dynamic allocation while keeping the memory flow visible. For the evaluated denoising autoencoder, we observed comparable reconstruction quality, a 1.87× reduction in inference latency, and a smaller PSRAM footprint.
9. Notes and limitations
The benchmark should be interpreted as a runtime comparison for this specific model and platform. TensorFlow Lite Micro was used with a beginner-friendly AllOpsResolver, which may include more operators than strictly necessary. A minimal resolver could reduce flash usage and possibly reduce some integration overhead. The tensor arena was tuned down to 32 KiB for the final memory comparison, but the exact minimum arena requirement may depend on the TFLite Micro version, operator implementations, memory planner, compiler, and build configuration.
Conversely, our Noodle result benefits from an explicit execution schedule tailored to this model. The newer NoodleBuffer abstraction reduces the user burden of manual buffer allocation, but the model still follows a hand-written sequence of operator calls. Therefore, we interpret the comparison as primarily demonstrating the benefit of explicit scheduling and direct operator execution for small embedded neural networks, rather than claiming universal superiority across all models or deployment scenarios.