Skip to content

simple_piano

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History
 
 

simple_piano

README.md

Simple Piano Example

This example demonstrates a complete polyphonic piano synthesizer using the esp_audio_render component. It showcases advanced audio generation, multi-track music composition, and real-time audio rendering capabilities.

🎹 Features Demonstrated

Core Audio Capabilities

  • Multi-track Audio Rendering: 4 independent audio streams (melody, harmony, bass, arpeggio)
  • Real-time Audio Generation: 20ms chunk-based processing for smooth playback
  • Stream Management: Independent stream control with proper lifecycle management

Advanced Piano Synthesis

  • Harmonic-rich Piano Tones: 4-harmonic synthesis (fundamental + 3 overtones)
  • ADSR Envelope: Professional attack, decay, sustain, release shaping
  • Polyphonic Support: Multiple notes playing simultaneously
  • Octave Range: Support for octaves 2-7 (C2 to B7)
  • Optimized Performance: Fast sine lookup tables with interpolation

Music Composition System

  • Multi-track Composition: Up to 4 independent musical tracks
  • Note-based Music: C, D, E, F, G, A, B note system with durations
  • Tempo Control: Configurable BPM and beat duration
  • Real-time Rendering: Dynamic note generation and mixing

Real-time Piano Control (Optional)

  • Interactive Piano Playing: When SUPPORT_REALTIME_TRACK is enabled
  • UART Control Interface: Real-time piano control via Python script
  • Press/Release Events: Natural piano key press and release behavior
  • Multi-octave Support: C3-B3 (low), C4-B4 (mid), C5-B5 (high) ranges
  • Live Performance: Play piano in real-time through keyboard input

🏗️ Architecture Overview

graph TB
    subgraph "Audio Rendering"
        H[Mixer] --> I[GMF Pipeline]
        I --> J[Audio Codec]
        J --> K[Speaker Output]
    end

    subgraph "Multi-track Mixing"
        L[Melody Track] --> H
        M[Harmony Track] --> H
        N[Bass Track] --> H
        O[Arpeggio Track] --> H
    end
Loading

🎵 How It Works

1. Music Definition

The example defines a complete song ("Twinkle Twinkle Little Star") with 4 tracks:

  • Track 0 (Melody): Main tune in octave 4
  • Track 1 (Harmony): Chord accompaniment in octave 3-4
  • Track 2 (Bass): Low bass line in octave 2
  • Track 3 (Arpeggio): Broken chord patterns in octave 4-5

2. Real-time Note Generation

// Process in 20ms chunks for smooth audio
const uint32_t chunk_duration_ms = 20;
uint32_t pcm_size = (sample_rate * chunk_duration_ms / 1000) * channels * 2;

// Generate audio for each track independently
for (int track_idx = 0; track_idx < MAX_TRACKS; track_idx++) {
    // Generate note audio for current time position
    piano_tone_gen_note_from_offset(tone_gen, note, octave,
                                   chunk_duration_ms, buffer, buffer_size,
                                   note_elapsed, full_note_duration);
}

3. Real-time Audio Rendering

Each of the generated track data is then fed into audio render streams using esp_audio_render_stream_write.

Audio render will mix all track data and finally output through codec devices.

4. Interactive Piano Control (with SUPPORT_REALTIME_TRACK)

When real-time piano is enabled, the system can receive UART commands for live piano playing.

The Python controller (piano_key.py) provides:

  • Terminal-based input: Raw keyboard capture without root permissions
  • Multi-octave support: Different octaves for different key ranges

🚀 Running the Example

Hardware Requirements

Software Requirements

  • ESP‑IDF v5.4 or later
  • esp_audio_render component
  • ESP‑GMF framework (format converters)
  • esp_codec_dev (audio output)

Build and Flash

# Navigate to example directory
cd examples/simple_piano

# Build and flash
idf.py -p /dev/ttyUSB0 flash monitor

Enable Real-time Piano (Optional)

To enable interactive piano control via UART:

  1. Rebuild after Enable the feature in piano_example.c:

    #define SUPPORT_REALTIME_TRACK

  2. Use the Python controller (in a separate terminal):

    # Install dependencies
pip install pyserial

# Run the piano controller
python3 piano_key.py --port /dev/ttyUSB0 --baud 115200

  3. Play piano in real-time:

    • Numbers 1-7: C4-B4 (mid octave)
    • Letters Q-U: C5-B5 (high octave)
    • ESC: Stop piano
    • Ctrl+C: Exit controller

🔧 Key API Usage

Piano Tone Generation

// Create piano tone generator
piano_tone_cfg_t cfg = {
    .sample_rate = 16000,
    .channels = 1,
    .bits_per_sample = 16,
    .attack_time = 0.01f,    // 10ms attack
    .decay_time = 0.05f,     // 50ms decay
    .sustain_level = 0.7f,   // 70% sustain
    .release_time = 0.1f     // 100ms release
};
piano_tone_gen_create(&cfg, &tone_gen);

// Generate note audio
piano_tone_gen_note_from_offset(tone_gen, NOTE_C, OCTAVE_4,
                               20, buffer, buffer_size,
                               elapsed_time, full_duration);

Song Rendering

// Create song renderer
song_cfg_t song_cfg = {
    .sample_rate = 16000,
    .channels = 1,
    .bits_per_sample = 16,
    .track_count = 4,
    .tempo = 120,              // 120 BPM
    .beat_duration_ms = 500    // 500ms per beat
};
song_render_create(&song_cfg, &song);

// Add musical tracks
song_render_add_track(song, 0, melody_notes, melody_octaves,
                     melody_durations, note_count);

// Start playback
song_render_play(song, audio_render);

Audio Render Integration

// Create audio renderer
esp_audio_render_create(&cfg, &audio_render);

// Open multiple streams
esp_audio_render_stream_handle_t stream_handles[4];
for (int i = 0; i < 4; i++) {
   esp_audio_render_stream_get(render, ESP_AUDIO_RENDER_STREAM_ID(i), &stream_handles[i]);
}
esp_audio_render_stream_open(stream_handles[0], &input_info);  // Melody
esp_audio_render_stream_open(stream_handles[1], &input_info);  // Harmony
esp_audio_render_stream_open(stream_handles[2], &input_info);  // Bass
esp_audio_render_stream_open(stream_handles[3], &input_info);  // Arpeggio

// Write audio data continuously
esp_audio_render_stream_handle_t current_stream = NULL;
esp_audio_render_stream_get(render->audio_render, ESP_AUDIO_RENDER_STREAM_ID(track->track_id), &current_stream);
esp_audio_render_stream_write(current_stream, pcm_buffer, pcm_size);

🎼 Music Theory Implementation

Note System

  • Western Scale: C, D, E, F, G, A, B (Do, Re, Mi, Fa, Sol, La, Si)
  • Frequency Calculation: freq = base_freq * 2^(octave - 4)
  • Base Frequencies: C4 = 261.63 Hz, D4 = 293.66 Hz, etc.

Harmonic Synthesis

Implemented in generate_piano_wave

ADSR Envelope

Implemented in calculate_adsr_envelope

🎯 Use Cases

Educational Applications

  • Music Theory Learning: Note recognition and frequency relationships
  • Composition Practice: Multi-track music creation
  • Audio Programming: DSP and synthesis concepts

Product Development

  • Audio Feedback: System sounds and notifications
  • Music Players: Embedded music synthesis
  • Interactive Audio: Real-time audio generation
  • Live Music Performance: Real-time piano playing and MIDI-like control

Testing and Validation

  • Audio Pipeline Testing: Multi-stream audio rendering
  • Performance Benchmarking: Real-time audio generation
  • Format Conversion: GMF pipeline validation

Other Board Support

This example can leverage gmf_app_utils for quick board bring‑up. Check board compatibility in menuconfig under “GMF APP Configuration.” For details, see the gmf_app_utils README.

Alternatively, you can use esp-bsp APIs:

  • Use bsp_audio_codec_microphone_init() to replace esp_gmf_app_get_record_handle()

🤝 Contributing

This example demonstrates best practices for:

  • Audio Synthesis: Efficient tone generation algorithms
  • Multi-track Audio: Independent stream management
  • Real-time Rendering: Chunk-based audio processing
  • Music Composition: Structured note-based music system

Feel free to extend this example with additional features or optimizations!