Rewrite MIDI input.

- A new "MIDI source" node, which can feed MIDI events from any ALSA device into the graph.
- All done rt safe in C++. The older Python implementation in noisicaa.devices has been removed.
- Still quite hacky:
  - It's built into the portaudio backend. There should probably be a separate "ALSA sequencer"
    backend for this, with the option to have an alternative "Jack MIDI" backend. But for that I
    have to figure out how to have separate backends for audio and MIDI and how that interacts
    for all possible combinations.
  - It just blindly connects to all readable MIDI ports, collecting all events. Filtering out
    the events for a specific port (e.g. a MIDI keyboard) happens in the "MIDI source" node.
    The engine should track, which ports are being used and only connect to those.

Squashed commit of the following:

commit c811be510347d1fd23abea081ba0a4d93e8cb6bf
Author: Ben Niemann <pink@odahoda.de>
Date:   Mon Jan 14 03:36:18 2019 +0100

    Move ALSADeviceManager to a separate file.

commit 6e5d9a2c691fdf639f0173b9dd2ebfde7f58f4f4
Author: Ben Niemann <pink@odahoda.de>
Date:   Mon Jan 14 03:25:29 2019 +0100

    Fix/improve tests.

commit 94b4fa253f8a4f8a84d13dd718dbaeac99fee5fe
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 13:57:07 2019 +0100

    Reanimate playback from the instrument library.

commit 17a288980fc361f190876763dbe4a6a6bbd0c2b3
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 12:57:54 2019 +0100

    Remove the now obsolete noisicaa.devices package.

commit aa2f9bbc1ae61295157f66948b276861dee00379
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 12:45:50 2019 +0100

    Strip the PianoWidget down to just the keys.

commit 1c87b29f7abb51defa28b33f902f8de85ae7eb55
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 12:24:57 2019 +0100

    Add piano to MIDI source node.

    - Make BasePipelineGraphNode.pipeline_node_id globally available.
    - Allow sending processor messages from the UI.
    - Pass the MIDI events to the rt thread via a FIFO queue.

commit f19114e966ab2d9261fd3a86b93d2ca88e9f3fba
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 11:29:43 2019 +0100

    Remove the System Out node again.

    And the related Backend::input().
    Not needed after all.

commit a839f259e3b8e338072be9c8b9fa58d8dc0d36a4
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 10:03:33 2019 +0100

    Wire up MIDI source to events from the backend.

    - Make the event buffer accessible via the block context.
    - Backend creates sequence of (uri, midi) tuples.
    - ProcessorMidiSource filters that list and emit list of midi events.

commit 347dc0168b00315eed233fdec40c8a9d6b5ffe41
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 09:21:36 2019 +0100

    Make the main ALSA sequences listen to all output ports again.

    Now also tracking new ports as they appear.

commit 86b6b7a59974c18c6078761fe1010456e5f26e43
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 09:20:52 2019 +0100

    Bug fix.

commit 776dbd4a946ecfa8e178cd7e3e108a4c3519f3cb
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 08:43:47 2019 +0100

    Editor tracks devices in a QAbstractItemModel.

    And MIDI Source node uses that for the port selector combobox.

commit a9c578e377948d187a0ee8ede90c29cc32b337a1
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 08:42:30 2019 +0100

    Also handle port changes somewhat gracefully.

commit f4cd8c7535b36e7c6b9323ff2861d72e376bac08
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 05:34:01 2019 +0100

    Also handle CC events.

commit 1329e51ff9747764a2bb5c6578f3490047cee135
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 04:30:45 2019 +0100

    Device manager that tracks ALSA sequencer clients.

    - Allow backends to post engine notifications.
    - PortAudioBackend runs a separate (non-rt) thread, with a sequencer client just for listening for
      client notifications.
    - Create a DeviceDescription proto for ALSA sequencer clients and post them as engine notifications.

commit 10c5b827de47479e6a8046c44cd32494693c762b
Author: Ben Niemann <pink@odahoda.de>
Date:   Sun Jan 13 01:34:47 2019 +0100

    A MIDI source node, which doesn't really do anything yet.

commit e09a5c70e3b950f3c6e30b81c2e8b67d65a947b3
Author: Ben Niemann <pink@odahoda.de>
Date:   Sat Jan 12 11:51:36 2019 +0100

    Use C string for Spec::get_buffer_idx() to avoid malloc in the audio thread.

commit 24cfffdf60a4ad888e65fe839165666ebef0f9f0
Author: Ben Niemann <pink@odahoda.de>
Date:   Sat Jan 12 08:13:26 2019 +0100

    Add a "System In" node to the graph and wire it up to the MIDI events from the backend.

    Also rename "Audio Out" to "System Out", because that makes more sense, now that there is more than
    audio being passed around.

commit 77be27b0e487b0830d913bdcc54cf56ea35114cf
Author: Ben Niemann <pink@odahoda.de>
Date:   Sat Jan 12 08:10:55 2019 +0100

    Add Backend::input() method to read incoming MIDI events.

    Also switch to an enum for the channel arg.

commit 5c4acefc476ace640d8a0ac40d6816ca48399207
Author: Ben Niemann <pink@odahoda.de>
Date:   Sat Jan 12 08:08:20 2019 +0100

    PortAudioBackend also reads MIDI events into a buffer.

    Very prototypish implementation. It just scans for all available devices and connects to their
    outputs. Still need to think about how to deal with different devices.
looper
Ben Niemann 4 years ago
parent e71cf92ba9
commit f4ce7f53d2

@ -32,7 +32,6 @@ add_python_package(
add_subdirectory(audioproc)
add_subdirectory(bindings)
add_subdirectory(core)
add_subdirectory(devices)
add_subdirectory(host_system)
add_subdirectory(instr)
add_subdirectory(instrument_db)

@ -47,4 +47,6 @@ from .public import (
ProcessorMessageList,
PlayerState,
TimeMapper,
DeviceDescription,
DevicePortDescription,
)

@ -48,6 +48,7 @@ add_python_package(
)
set(LIB_SRCS
alsa_device_manager.cpp
backend.cpp
backend_null.cpp
backend_portaudio.cpp

@ -0,0 +1,249 @@
/*
* @begin:license
*
* Copyright (c) 2015-2019, Benjamin Niemann <pink@odahoda.de>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*
* @end:license
*/
#include <google/protobuf/util/message_differencer.h>
#include "noisicaa/core/logging.h"
#include "noisicaa/core/scope_guard.h"
#include "noisicaa/core/slots.inl.h"
#include "noisicaa/host_system/host_system.h"
#include "noisicaa/audioproc/public/devices.pb.h"
#include "noisicaa/audioproc/public/engine_notification.pb.h"
#include "noisicaa/audioproc/engine/misc.h"
#include "noisicaa/audioproc/engine/alsa_device_manager.h"
namespace noisicaa {
ALSADeviceManager::ALSADeviceManager(
int client_id,
Slot<pb::EngineNotification>& notifications)
: _logger(LoggerRegistry::get_logger("noisicaa.audioproc.engine.backend.alsa_device_manager")),
_client_id(client_id),
_notifications(notifications) {}
ALSADeviceManager::~ALSADeviceManager() {
for (const auto& it : _devices) {
remove_device(it.second);
}
_devices.clear();
if (_seq != nullptr) {
snd_seq_close(_seq);
_seq = nullptr;
}
}
Status ALSADeviceManager::setup() {
RETURN_IF_ALSA_ERROR(snd_seq_open(&_seq, "default", SND_SEQ_OPEN_DUPLEX, SND_SEQ_NONBLOCK));
RETURN_IF_ALSA_ERROR(snd_seq_set_client_name(_seq, "noisicaa device monitor"));
snd_seq_port_info_t *pinfo;
snd_seq_port_info_alloca(&pinfo);
snd_seq_port_info_set_capability(pinfo, SND_SEQ_PORT_CAP_WRITE | SND_SEQ_PORT_CAP_NO_EXPORT);
snd_seq_port_info_set_type(pinfo, SND_SEQ_PORT_TYPE_APPLICATION);
snd_seq_port_info_set_name(pinfo, "Input");
RETURN_IF_ALSA_ERROR(snd_seq_create_port(_seq, pinfo));
int input_port_id = snd_seq_port_info_get_port(pinfo);
// Connect to System Announce port
RETURN_IF_ALSA_ERROR(
snd_seq_connect_from(
_seq, input_port_id, SND_SEQ_CLIENT_SYSTEM, SND_SEQ_PORT_SYSTEM_ANNOUNCE));
snd_seq_client_info_t *cinfo;
snd_seq_client_info_alloca(&cinfo);
snd_seq_client_info_set_client(cinfo, -1);
while (snd_seq_query_next_client(_seq, cinfo) == 0) {
int client_id = snd_seq_client_info_get_client(cinfo);
if (client_id == snd_seq_client_id(_seq)
|| client_id == _client_id
|| client_id == SND_SEQ_CLIENT_SYSTEM) {
continue;
}
StatusOr<pb::DeviceDescription> stor_device = get_device_description(client_id);
RETURN_IF_ERROR(stor_device);
pb::DeviceDescription device = stor_device.result();
add_device(device);
_devices[device.uri()] = device;
}
return Status::Ok();
}
StatusOr<pb::DeviceDescription> ALSADeviceManager::get_device_description(int client_id) {
snd_seq_client_info_t *cinfo;
snd_seq_client_info_alloca(&cinfo);
snd_seq_port_info_t *pinfo;
snd_seq_port_info_alloca(&pinfo);
RETURN_IF_ALSA_ERROR(snd_seq_get_any_client_info(_seq, client_id, cinfo));
pb::DeviceDescription device;
device.set_uri(sprintf("alsa://%d", client_id));
device.set_type(pb::DeviceDescription::MIDI_CONTROLLER);
device.set_display_name(snd_seq_client_info_get_name(cinfo));
snd_seq_port_info_set_client(pinfo, client_id);
snd_seq_port_info_set_port(pinfo, -1);
while (snd_seq_query_next_port(_seq, pinfo) == 0) {
unsigned int cap = snd_seq_port_info_get_capability(pinfo);
if (cap & SND_SEQ_PORT_CAP_NO_EXPORT) {
continue;
}
pb::DevicePortDescription* port = device.add_ports();
int port_id = snd_seq_port_info_get_port(pinfo);
port->set_uri(sprintf("alsa://%d/%d", client_id, port_id));
port->set_display_name(snd_seq_port_info_get_name(pinfo));
if (cap & (SND_SEQ_PORT_CAP_READ | SND_SEQ_PORT_CAP_DUPLEX)) {
port->set_readable(true);
}
if (cap & (SND_SEQ_PORT_CAP_WRITE | SND_SEQ_PORT_CAP_DUPLEX)) {
port->set_writable(true);
}
}
return device;
}
void ALSADeviceManager::add_device(const pb::DeviceDescription& device) {
_logger->info("Added device:\n%s", device.DebugString().c_str());
pb::EngineNotification notification;
pb::DeviceManagerMessage* m = notification.add_device_manager_messages();
m->mutable_added()->CopyFrom(device);
_notifications.emit(notification);
}
void ALSADeviceManager::update_device(const pb::DeviceDescription& device) {
_logger->info("Updated device:\n%s", device.DebugString().c_str());
pb::EngineNotification notification;
pb::DeviceManagerMessage* m = notification.add_device_manager_messages();
m->mutable_removed()->CopyFrom(device);
m = notification.add_device_manager_messages();
m->mutable_added()->CopyFrom(device);
_notifications.emit(notification);
}
void ALSADeviceManager::remove_device(const pb::DeviceDescription& device) {
_logger->info("Removed device:\n%s", device.DebugString().c_str());
pb::EngineNotification notification;
pb::DeviceManagerMessage* m = notification.add_device_manager_messages();
m->mutable_removed()->CopyFrom(device);
_notifications.emit(notification);
}
void ALSADeviceManager::process_events() {
while (true) {
snd_seq_event_t* event;
int rc = snd_seq_event_input(_seq, &event);
if (rc == -ENOSPC) {
_logger->warning("ALSA midi queue overrun.");
return;
}
if (rc == -EAGAIN) {
return;
}
if (rc < 0) {
_logger->error("ALSA error %d: %s", rc, snd_strerror(rc));
return;
}
switch (event->type) {
case SND_SEQ_EVENT_PORT_START:
case SND_SEQ_EVENT_PORT_CHANGE:
case SND_SEQ_EVENT_PORT_EXIT:
case SND_SEQ_EVENT_CLIENT_START:
case SND_SEQ_EVENT_CLIENT_CHANGE: {
if (event->data.addr.client == snd_seq_client_id(_seq)
|| event->data.addr.client == _client_id
|| event->data.addr.client == SND_SEQ_CLIENT_SYSTEM) {
break;
}
StatusOr<pb::DeviceDescription> stor_device = get_device_description(event->data.addr.client);
if (stor_device.is_error()) {
_logger->error(
"Failed to get device description for ALSA sequencer client %d",
event->data.addr.client);
} else {
pb::DeviceDescription device = stor_device.result();
auto it = _devices.find(device.uri());
if (it == _devices.end()) {
add_device(device);
_devices[device.uri()] = device;
} else if (!google::protobuf::util::MessageDifferencer::Equals(device, it->second)) {
update_device(device);
_devices[device.uri()] = device;
}
}
break;
}
case SND_SEQ_EVENT_CLIENT_EXIT: {
if (event->data.addr.client == snd_seq_client_id(_seq)
|| event->data.addr.client == _client_id
|| event->data.addr.client == SND_SEQ_CLIENT_SYSTEM) {
break;
}
string uri = sprintf("alsa://%d", event->data.addr.client);
auto it = _devices.find(uri);
if (it == _devices.end()) {
_logger->warning("Got CLIENT_EXIT event for unknown client.");
} else {
remove_device(it->second);
_devices.erase(it);
}
break;
}
case SND_SEQ_EVENT_PORT_SUBSCRIBED:
case SND_SEQ_EVENT_PORT_UNSUBSCRIBED:
// Ignore these events.
break;
default:
_logger->error(
"Unknown MIDI event: type=%d flags=%x tag=%x queue=%x time=%d source=%d.%d dest=%d.%d",
event->type,
event->flags,
event->tag,
event->queue,
event->time.tick,
event->source.client,
event->source.port,
event->dest.client,
event->dest.port);
break;
}
}
}
} // namespace noisicaa

@ -0,0 +1,68 @@
// -*- mode: c++ -*-
/*
* @begin:license
*
* Copyright (c) 2015-2019, Benjamin Niemann <pink@odahoda.de>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc.,
* 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*
* @end:license
*/
#ifndef _NOISICAA_AUDIOPROC_ENGINE_ALSA_DEVICE_MANAGER_H
#define _NOISICAA_AUDIOPROC_ENGINE_ALSA_DEVICE_MANAGER_H
#include <map>
#include <string>
#include "alsa/asoundlib.h"
#include "noisicaa/core/slots.h"
#include "noisicaa/core/status.h"
namespace noisicaa {
class Logger;
namespace pb {
class EngineNotification;
class DeviceDescription;
}
class ALSADeviceManager {
public:
ALSADeviceManager(
int client_id,
Slot<pb::EngineNotification>& notifications);
~ALSADeviceManager();
Status setup();
void process_events();
private:
StatusOr<pb::DeviceDescription> get_device_description(int client_id);
void add_device(const pb::DeviceDescription& device);
void update_device(const pb::DeviceDescription& device);
void remove_device(const pb::DeviceDescription& device);
private:
Logger* _logger;
int _client_id;
Slot<pb::EngineNotification>& _notifications;
snd_seq_t* _seq = nullptr;
map<string, pb::DeviceDescription> _devices;
};
} // namespace noisicaa
#endif

@ -20,6 +20,7 @@
* @end:license
*/
#include "noisicaa/audioproc/public/engine_notification.pb.h"
#include "noisicaa/audioproc/engine/backend.h"
#include "noisicaa/audioproc/engine/backend_null.h"
#include "noisicaa/audioproc/engine/backend_portaudio.h"
@ -27,28 +28,43 @@
namespace noisicaa {
Backend::Backend(HostSystem* host_system, const char* logger_name, const BackendSettings& settings)
Backend::Backend(
HostSystem* host_system, const char* logger_name, const BackendSettings& settings,
void (*callback)(void*, const string&), void *userdata)
: _host_system(host_system),
_logger(LoggerRegistry::get_logger(logger_name)),
_settings(settings) {}
_settings(settings),
_callback(callback),
_userdata(userdata) {
notifications.connect(std::bind(&Backend::notification_proxy, this, placeholders::_1));
}
Backend::~Backend() {
cleanup();
}
StatusOr<Backend*> Backend::create(
HostSystem* host_system, const string& name, const BackendSettings& settings) {
HostSystem* host_system, const string& name, const BackendSettings& settings,
void (*callback)(void*, const string&), void* userdata) {
if (name == "portaudio") {
return new PortAudioBackend(host_system, settings);
return new PortAudioBackend(host_system, settings, callback, userdata);
} else if (name == "null") {
return new NullBackend(host_system, settings);
return new NullBackend(host_system, settings, callback, userdata);
} else if (name == "renderer") {
return new RendererBackend(host_system, settings);
return new RendererBackend(host_system, settings, callback, userdata);
}
return ERROR_STATUS("Invalid backend name '%s'", name.c_str());
}
void Backend::notification_proxy(const pb::EngineNotification& notification) {
if (_callback != nullptr) {
string notification_serialized;
assert(notification.SerializeToString(&notification_serialized));
_callback(_userdata, notification_serialized);
}
}
Status Backend::setup(Realm* realm) {
_realm = realm;
return Status::Ok();

@ -27,6 +27,7 @@
#include <string>
#include "noisicaa/core/logging.h"
#include "noisicaa/core/slots.h"
#include "noisicaa/core/status.h"
#include "noisicaa/audioproc/engine/buffers.h"
@ -37,6 +38,9 @@ using namespace std;
class BlockContext;
class Realm;
class HostSystem;
namespace pb {
class EngineNotification;
}
struct BackendSettings {
string datastream_address;
@ -45,24 +49,39 @@ struct BackendSettings {
class Backend {
public:
enum Channel {
AUDIO_LEFT = 1,
AUDIO_RIGHT = 2,
EVENTS = 3,
};
virtual ~Backend();
Slot<pb::EngineNotification> notifications;
static StatusOr<Backend*> create(
HostSystem* host_system, const string& name, const BackendSettings& settings);
HostSystem* host_system, const string& name, const BackendSettings& settings,
void (*callback)(void*, const string&), void* userdata);
virtual Status setup(Realm* realm);
virtual void cleanup();
virtual Status begin_block(BlockContext* ctxt) = 0;
virtual Status end_block(BlockContext* ctxt) = 0;
virtual Status output(BlockContext* ctxt, const string& channel, BufferPtr samples) = 0;
virtual Status output(BlockContext* ctxt, Channel channel, BufferPtr buffer) = 0;
protected:
Backend(HostSystem* host_system, const char* logger_name, const BackendSettings& settings);
Backend(
HostSystem* host_system, const char* logger_name, const BackendSettings& settings,
void (*callback)(void*, const string&), void* userdata);
void notification_proxy(const pb::EngineNotification& notification);
HostSystem* _host_system;
Logger* _logger;
BackendSettings _settings;
void (*_callback)(void*, const string&);
void *_userdata;
Realm* _realm = nullptr;
};

@ -35,15 +35,21 @@ cdef extern from "noisicaa/audioproc/engine/backend.h" namespace "noisicaa" nogi
float time_scale
cppclass Backend:
enum Channel:
AUDIO_LEFT "noisicaa::Backend::AUDIO_LEFT"
AUDIO_RIGHT "noisicaa::Backend::AUDIO_RIGHT"
EVENTS "noisicaa::Backend::EVENTS"
@staticmethod
StatusOr[Backend*] create(
HostSystem* host_system, const string& name, const BackendSettings& settings)
HostSystem* host_system, const string& name, const BackendSettings& settings,
void (*callback)(void*, const string&), void* userdata)
Status setup(Realm* realm)
void cleanup()
Status begin_block(BlockContext* ctxt)
Status end_block(BlockContext* ctxt)
Status output(BlockContext* ctxt, const string& channel, BufferPtr samples)
Status output(BlockContext* ctxt, Channel channel, BufferPtr samples)
cdef class PyBackendSettings(object):
@ -55,5 +61,8 @@ cdef class PyBackendSettings(object):
cdef class PyBackend(object):
cdef unique_ptr[Backend] __backend_ptr
cdef Backend* __backend
cdef readonly object notifications
cdef Backend* get(self) nogil
@staticmethod
cdef void __notification_callback(void* c_self, const string& notification_serialized) with gil

@ -19,9 +19,13 @@
# @end:license
from libc.stdint cimport uint8_t
from cpython.ref cimport PyObject
from cpython.exc cimport PyErr_Fetch, PyErr_Restore
from noisicaa import core
from noisicaa.core.status cimport check
from noisicaa.host_system.host_system cimport PyHostSystem
from noisicaa.audioproc.public import engine_notification_pb2
from . cimport block_context
from . cimport buffers
@ -59,11 +63,15 @@ cdef class PyBackendSettings(object):
cdef class PyBackend(object):
def __init__(self, PyHostSystem host_system, name, PyBackendSettings settings):
self.notifications = core.Callback()
if isinstance(name, str):
name = name.encode('ascii')
assert isinstance(name, bytes)
cdef StatusOr[Backend*] backend = Backend.create(host_system.get(), name, settings.get())
cdef StatusOr[Backend*] backend = Backend.create(
host_system.get(), name, settings.get(),
self.__notification_callback, <PyObject*>self)
check(backend)
self.__backend_ptr.reset(backend.result())
self.__backend = self.__backend_ptr.get()
@ -71,6 +79,25 @@ cdef class PyBackend(object):
cdef Backend* get(self) nogil:
return self.__backend
@staticmethod
cdef void __notification_callback(void* c_self, const string& notification_serialized) with gil:
self = <object><PyObject*>c_self
# Have to stash away any active exception, because otherwise exception handling
# might get confused.
# See https://github.com/cython/cython/issues/1877
cdef PyObject* exc_type
cdef PyObject* exc_value
cdef PyObject* exc_trackback
PyErr_Fetch(&exc_type, &exc_value, &exc_trackback)
try:
notification = engine_notification_pb2.EngineNotification()
notification.ParseFromString(notification_serialized)
self.notifications.call(notification)
finally:
PyErr_Restore(exc_type, exc_value, exc_trackback)
def setup(self, PyRealm realm):
cdef Realm* c_realm = realm.get()
with nogil:
@ -93,6 +120,14 @@ cdef class PyBackend(object):
def output(self, block_context.PyBlockContext ctxt, str channel, float[:] samples):
cdef buffers.BufferPtr c_samples = <BufferPtr>&samples[0]
cdef string c_channel = channel.encode('utf-8')
cdef Backend.Channel c_channel
if channel == 'left':
c_channel = Backend.Channel.AUDIO_LEFT
elif channel == 'right':
c_channel = Backend.Channel.AUDIO_RIGHT
elif channel == 'events':
c_channel = Backend.Channel.EVENTS
else:
raise ValueError(channel)
with nogil:
check(self.__backend.output(ctxt.get(), c_channel, c_samples))

@ -29,8 +29,10 @@
namespace noisicaa {
NullBackend::NullBackend(HostSystem* host_system, const BackendSettings& settings)
: Backend(host_system, "noisicaa.audioproc.engine.backend.null", settings) {}
NullBackend::NullBackend(
HostSystem* host_system, const BackendSettings& settings,
void (*callback)(void*, const string&), void *userdata)
: Backend(host_system, "noisicaa.audioproc.engine.backend.null", settings, callback, userdata) {}
NullBackend::~NullBackend() {}
@ -68,7 +70,7 @@ Status NullBackend::end_block(BlockContext* ctxt) {
return Status::Ok();
}
Status NullBackend::output(BlockContext* ctxt, const string& channel, BufferPtr samples) {
Status NullBackend::output(BlockContext* ctxt, Channel channel, BufferPtr buffer) {
return Status::Ok();
}

@ -37,7 +37,9 @@ class Realm;
class NullBackend : public Backend {
public:
NullBackend(HostSystem* host_system, const BackendSettings& settings);
NullBackend(
HostSystem* host_system, const BackendSettings& settings,
void (*callback)(void*, const string&), void* userdata);
~NullBackend() override;
Status setup(Realm* realm) override;
@ -45,7 +47,7 @@ class NullBackend : public Backend {
Status begin_block(BlockContext* ctxt) override;
Status end_block(BlockContext* ctxt) override;
Status output(BlockContext* ctxt, const string& channel, BufferPtr samples) override;
Status output(BlockContext* ctxt, Channel channel, BufferPtr buffer) override;
private:
chrono::time_point<std::chrono::high_resolution_clock> _block_start;

@ -20,19 +20,29 @@
* @end:license
*/
#include <google/protobuf/util/message_differencer.h>
#include "noisicaa/core/perf_stats.h"
#include "noisicaa/core/scope_guard.h"
#include "noisicaa/host_system/host_system.h"
#include "noisicaa/audioproc/public/devices.pb.h"
#include "noisicaa/audioproc/public/engine_notification.pb.h"
#include "noisicaa/audioproc/engine/backend_portaudio.h"
#include "noisicaa/audioproc/engine/realm.h"
#include "noisicaa/audioproc/engine/rtcheck.h"
#include "noisicaa/audioproc/engine/alsa_device_manager.h"
namespace noisicaa {
PortAudioBackend::PortAudioBackend(HostSystem* host_system, const BackendSettings& settings)
: Backend(host_system, "noisicaa.audioproc.engine.backend.portaudio", settings),
PortAudioBackend::PortAudioBackend(
HostSystem* host_system, const BackendSettings& settings,
void (*callback)(void*, const string&), void *userdata)
: Backend(host_system, "noisicaa.audioproc.engine.backend.portaudio", settings, callback, userdata),
_initialized(false),
_stream(nullptr),
_samples{nullptr, nullptr} {
_samples{nullptr, nullptr},
_seq(nullptr),
_events(nullptr) {
}
PortAudioBackend::~PortAudioBackend() {}
@ -48,10 +58,77 @@ Status PortAudioBackend::setup(Realm* realm) {
RETURN_IF_ERROR(setup_stream());
_device_thread_stop.exchange(false);
StatusSignal status;
_device_thread.reset(new thread(&PortAudioBackend::device_thread_main, this, &status));
RETURN_IF_ERROR(status.wait());
RETURN_IF_ALSA_ERROR(snd_seq_open(&_seq, "default", SND_SEQ_OPEN_DUPLEX, SND_SEQ_NONBLOCK));
RETURN_IF_ALSA_ERROR(snd_seq_set_client_name(_seq, "noisicaa"));
_client_id = snd_seq_client_id(_seq);
snd_seq_port_info_t *pinfo;
snd_seq_port_info_alloca(&pinfo);
snd_seq_port_info_set_capability(pinfo, SND_SEQ_PORT_CAP_WRITE);
snd_seq_port_info_set_type(
pinfo, SND_SEQ_PORT_TYPE_MIDI_GENERIC | SND_SEQ_PORT_TYPE_APPLICATION);
snd_seq_port_info_set_name(pinfo, "Input");
RETURN_IF_ALSA_ERROR(snd_seq_create_port(_seq, pinfo));
_input_port_id = snd_seq_port_info_get_port(pinfo);
// Connect to System Announce port
RETURN_IF_ALSA_ERROR(
snd_seq_connect_from(
_seq, _input_port_id, SND_SEQ_CLIENT_SYSTEM, SND_SEQ_PORT_SYSTEM_ANNOUNCE));
snd_seq_client_info_t *cinfo;
snd_seq_client_info_alloca(&cinfo);
snd_seq_client_info_set_client(cinfo, -1);
while (snd_seq_query_next_client(_seq, cinfo) == 0) {
int client_id = snd_seq_client_info_get_client(cinfo);
if (client_id == snd_seq_client_id(_seq) || client_id == SND_SEQ_CLIENT_SYSTEM) {
continue;
}
snd_seq_port_info_t *pinfo;
snd_seq_port_info_alloca(&pinfo);
snd_seq_port_info_set_client(pinfo, client_id);
snd_seq_port_info_set_port(pinfo, -1);
while (snd_seq_query_next_port(_seq, pinfo) == 0) {
int port_id = snd_seq_port_info_get_port(pinfo);
unsigned int cap = snd_seq_port_info_get_capability(pinfo);
if (cap & SND_SEQ_PORT_CAP_READ && !(cap & SND_SEQ_PORT_CAP_NO_EXPORT)) {
RETURN_IF_ALSA_ERROR(
snd_seq_connect_from(_seq, _input_port_id, client_id, port_id));
_logger->info(
"Listening to MIDI sequencer port %d.%d",
client_id, port_id);
}
}
}
_events = new uint8_t[10240];
return Status::Ok();
}
void PortAudioBackend::cleanup() {
if (_events != nullptr) {
delete _events;
_events = nullptr;
}
if (_seq != nullptr) {
snd_seq_close(_seq);
_seq = nullptr;
}
if (_device_thread.get() != nullptr) {
_device_thread_stop = true;
_device_thread->join();
_device_thread.reset();
}
cleanup_stream();
if (_initialized) {
@ -124,6 +201,28 @@ void PortAudioBackend::cleanup_stream() {
}
}
void PortAudioBackend::device_thread_main(StatusSignal* status) {
_logger->info("Starting ALSA device listener thread...");
auto goodbye = scopeGuard([this]() {
_logger->info("ALSA device listener thread stopped");
});
ALSADeviceManager mgr(_client_id, notifications);
Status mgr_status = mgr.setup();
if (mgr_status.is_error()) {
status->set(mgr_status);
return;
}
status->set(Status::Ok());
while (!_device_thread_stop.load()) {
std::this_thread::sleep_for(std::chrono::milliseconds(10));
mgr.process_events();
}
}
Status PortAudioBackend::begin_block(BlockContext* ctxt) {
assert(ctxt->perf->current_span_id() == 0);
ctxt->perf->start_span("frame");
@ -132,6 +231,163 @@ Status PortAudioBackend::begin_block(BlockContext* ctxt) {
memset(_samples[c], 0, _host_system->block_size() * sizeof(float));
}
LV2_Atom_Forge forge;
lv2_atom_forge_init(&forge, &_host_system->lv2->urid_map);
LV2_Atom_Forge_Frame frame;
lv2_atom_forge_set_buffer(&forge, _events, 10240);
lv2_atom_forge_sequence_head(&forge, &frame, _host_system->lv2->urid.atom_frame_time);
while (true) {
snd_seq_event_t* event;
int rc = snd_seq_event_input(_seq, &event);
if (rc == -ENOSPC) {
_logger->warning("ALSA midi queue overrun.");
break;
}
if (rc == -EAGAIN) {
break;
}
RETURN_IF_ALSA_ERROR(rc);
if ((event->flags & SND_SEQ_TIME_STAMP_MASK) != SND_SEQ_TIME_STAMP_TICK) {
_logger->error("Event without tick");
continue;
}
switch (event->type) {
case SND_SEQ_EVENT_NOTEON: {
_logger->debug(
"Note on: time=%d source=%d.%d channel=%d note=%d velocity=%d",
event->time.tick,
event->source.client, event->source.port,
event->data.note.channel,
event->data.note.note,
event->data.note.velocity);
char uri[128];
snprintf(uri, sizeof(uri), "alsa://%d/%d", event->source.client, event->source.port);
uint8_t msg[3];
msg[0] = 0x90 | event->data.note.channel;
msg[1] = event->data.note.note;
msg[2] = event->data.note.velocity;
lv2_atom_forge_frame_time(&forge, 0);
LV2_Atom_Forge_Frame tframe;
lv2_atom_forge_tuple(&forge, &tframe);
lv2_atom_forge_string(&forge, uri, strlen(uri));
lv2_atom_forge_atom(&forge, 3, _host_system->lv2->urid.midi_event);
lv2_atom_forge_write(&forge, msg, 3);
lv2_atom_forge_pop(&forge, &tframe);
break;
}
case SND_SEQ_EVENT_NOTEOFF: {
_logger->debug(
"Note off: time=%d source=%d.%d channel=%d note=%d velocity=%d",
event->time.tick,
event->source.client, event->source.port,
event->data.note.channel,
event->data.note.note,
event->data.note.velocity);
char uri[128];
snprintf(uri, sizeof(uri), "alsa://%d/%d", event->source.client, event->source.port);
uint8_t msg[3];
msg[0] = 0x80 | event->data.note.channel;
msg[1] = event->data.note.note;
msg[2] = event->data.note.velocity;
lv2_atom_forge_frame_time(&forge, 0);
LV2_Atom_Forge_Frame tframe;
lv2_atom_forge_tuple(&forge, &tframe);
lv2_atom_forge_string(&forge, uri, strlen(uri));
lv2_atom_forge_atom(&forge, 3, _host_system->lv2->urid.midi_event);
lv2_atom_forge_write(&forge, msg, 3);
lv2_atom_forge_pop(&forge, &tframe);
break;
}
case SND_SEQ_EVENT_CONTROLLER:
_logger->debug(
"CC: time=%d source=%d.%d channel=%d, param=%d value=%d",
event->time.tick,
event->source.client, event->source.port,
event->data.control.channel,
event->data.control.param,
event->data.control.value);
char uri[128];
snprintf(uri, sizeof(uri), "alsa://%d/%d", event->source.client, event->source.port);
uint8_t msg[3];
msg[0] = 0xa0 | event->data.control.channel;
msg[1] = event->data.control.param;
msg[2] = event->data.control.value;
lv2_atom_forge_frame_time(&forge, 0);
LV2_Atom_Forge_Frame tframe;
lv2_atom_forge_tuple(&forge, &tframe);
lv2_atom_forge_string(&forge, uri, strlen(uri));
lv2_atom_forge_atom(&forge, 3, _host_system->lv2->urid.midi_event);
lv2_atom_forge_write(&forge, msg, 3);
lv2_atom_forge_pop(&forge, &tframe);
break;
case SND_SEQ_EVENT_PORT_START: {
snd_seq_port_info_t *pinfo;
snd_seq_port_info_alloca(&pinfo);
int rc = snd_seq_get_any_port_info(
_seq, event->data.addr.client, event->data.addr.port, pinfo);
if (rc < 0) {
_logger->error("ALSA error %d: %s", rc, snd_strerror(rc));
} else {
unsigned int cap = snd_seq_port_info_get_capability(pinfo);
if (cap & SND_SEQ_PORT_CAP_READ && !(cap & SND_SEQ_PORT_CAP_NO_EXPORT)) {
rc = snd_seq_connect_from(
_seq, _input_port_id, event->data.addr.client, event->data.addr.port);
if (rc < 0) {
_logger->error("ALSA error %d: %s", rc, snd_strerror(rc));
} else {
_logger->info(
"Listening to MIDI sequencer port %d.%d",
event->data.addr.client, event->data.addr.port);
}
}
}
break;
}
case SND_SEQ_EVENT_PORT_CHANGE:
case SND_SEQ_EVENT_PORT_EXIT:
case SND_SEQ_EVENT_CLIENT_START:
case SND_SEQ_EVENT_CLIENT_CHANGE:
case SND_SEQ_EVENT_CLIENT_EXIT:
case SND_SEQ_EVENT_PORT_SUBSCRIBED:
case SND_SEQ_EVENT_PORT_UNSUBSCRIBED:
// Ignore these events.
break;
default:
_logger->error(
"Unknown MIDI event: type=%d flags=%x tag=%x queue=%x time=%d source=%d.%d dest=%d.%d",
event->type,
event->flags,
event->tag,
event->queue,
event->time.tick,
event->source.client,
event->source.port,
event->dest.client,
event->dest.port);
}
}
lv2_atom_forge_pop(&forge, &frame);
ctxt->input_events = (LV2_Atom_Sequence*)_events;
return Status::Ok();
}
@ -150,15 +406,17 @@ Status PortAudioBackend::end_block(BlockContext* ctxt) {
return Status::Ok();
}
Status PortAudioBackend::output(BlockContext* ctxt, const string& channel, BufferPtr samples) {
if (channel == "left") {
Status PortAudioBackend::output(BlockContext* ctxt, Channel channel, BufferPtr samples) {
switch (channel) {
case AUDIO_LEFT:
memmove(_samples[0], samples, _host_system->block_size() * sizeof(float));
} else if (channel == "right") {
return Status::Ok();
case AUDIO_RIGHT:
memmove(_samples[1], samples, _host_system->block_size() * sizeof(float));
} else {
return ERROR_STATUS("Invalid channel %s", channel.c_str());
return Status::Ok();
default:
return ERROR_STATUS("Invalid channel %d", channel);
}
return Status::Ok();
}
} // namespace noisicaa

@ -25,8 +25,12 @@
#ifndef _NOISICAA_AUDIOPROC_ENGINE_BACKEND_PORTAUDIO_H
#define _NOISICAA_AUDIOPROC_ENGINE_BACKEND_PORTAUDIO_H
#include <atomic>
#include <memory>
#include <thread>
#include <string>
#include <stdint.h>
#include "alsa/asoundlib.h"
#include "portaudio.h"
#include "noisicaa/audioproc/engine/backend.h"
#include "noisicaa/audioproc/engine/buffers.h"
@ -37,7 +41,9 @@ class Realm;
class PortAudioBackend : public Backend {
public:
PortAudioBackend(HostSystem* host_system, const BackendSettings& settings);
PortAudioBackend(
HostSystem* host_system, const BackendSettings& settings,
void (*callback)(void*, const string&), void* userdata);
~PortAudioBackend() override;
Status setup(Realm* realm) override;
@ -45,7 +51,7 @@ public:
Status begin_block(BlockContext* ctxt) override;
Status end_block(BlockContext* ctxt) override;
Status output(BlockContext* ctxt, const string& channel, BufferPtr samples) override;
Status output(BlockContext* ctxt, Channel channel, BufferPtr samples) override;
private:
Status setup_stream();
@ -54,6 +60,15 @@ public:
bool _initialized;
PaStream* _stream;
BufferPtr _samples[2];
snd_seq_t* _seq;
int _client_id;
int _input_port_id;
BufferPtr _events;
unique_ptr<thread> _device_thread;
atomic<bool> _device_thread_stop;
void device_thread_main(StatusSignal* status);
};
} // namespace noisicaa

@ -36,8 +36,11 @@ extern "C" {
namespace noisicaa {
RendererBackend::RendererBackend(HostSystem* host_system, const BackendSettings& settings)
: Backend(host_system, "noisicaa.audioproc.engine.backend.renderer", settings) {}
RendererBackend::RendererBackend(
HostSystem* host_system, const BackendSettings& settings,
void (*callback)(void*, const string&), void *userdata)
: Backend(
host_system, "noisicaa.audioproc.engine.backend.renderer", settings, callback, userdata) {}
RendererBackend::~RendererBackend() {}
@ -138,21 +141,19 @@ Status RendererBackend::end_block(BlockContext* ctxt) {
return Status::Ok();
}
Status RendererBackend::output(BlockContext* ctxt, const string& channel, BufferPtr samples) {
Status RendererBackend::output(BlockContext* ctxt, Channel channel, BufferPtr buffer) {
int c;
if (channel == "left") {
c = 0;
} else if (channel == "right") {
c = 1;
} else {
return ERROR_STATUS("Invalid channel %s", channel.c_str());
switch (channel) {
case AUDIO_LEFT: c = 0; break;
case AUDIO_RIGHT: c = 1; break;
default: return ERROR_STATUS("Invalid channel %d", channel);
}
if (_channel_written[c]) {
return ERROR_STATUS("Channel %s written multiple times.", channel.c_str());
return ERROR_STATUS("Channel %d written multiple times.", c);
}
_channel_written[c] = true;
memmove(_samples[c].get(), samples, _host_system->block_size() * sizeof(float));
memmove(_samples[c].get(), buffer, _host_system->block_size() * sizeof(float));
return Status::Ok();
}

@ -37,7 +37,9 @@ class Realm;
class RendererBackend : public Backend {
public:
RendererBackend(HostSystem* host_system, const BackendSettings& settings);
RendererBackend(
HostSystem* host_system, const BackendSettings& settings,
void (*callback)(void*, const string&), void* userdata);
~RendererBackend() override;
Status setup(Realm* realm) override;
@ -45,7 +47,7 @@ public:
Status begin_block(BlockContext* ctxt) override;
Status end_block(BlockContext* ctxt) override;
Status output(BlockContext* ctxt, const string& channel, BufferPtr samples) override;
Status output(BlockContext* ctxt, Channel channel, BufferPtr buffer) override;
private:
unique_ptr<BufferData> _samples[2];

@ -30,6 +30,9 @@
#include <memory>
#include <string>
#include <vector>
#include "lv2/lv2plug.in/ns/ext/atom/atom.h"
#include "noisicaa/audioproc/public/musical_time.h"
#include "noisicaa/audioproc/engine/buffers.h"
@ -57,13 +60,7 @@ struct BlockContext {
void alloc_time_map(uint32_t block_size);
BufferArena* buffer_arena;
struct Buffer {
size_t size;
const BufferPtr data;
};
map<string, Buffer> buffers;
LV2_Atom_Sequence* input_events;
MessageQueue* out_messages;
};

@ -24,6 +24,7 @@ from libcpp.vector cimport vector
from noisicaa.core.perf_stats cimport PyPerfStats, PerfStats
from noisicaa.audioproc.public.musical_time cimport MusicalTime
from noisicaa.lv2.atom cimport LV2_Atom_Sequence
from .message_queue cimport MessageQueue
from .buffer_arena cimport BufferArena
@ -40,6 +41,7 @@ cdef extern from "noisicaa/audioproc/engine/block_context.h" namespace "noisicaa
unique_ptr[PerfStats] perf
MessageQueue* out_messages
BufferArena* buffer_arena
LV2_Atom_Sequence* input_events
cdef class PyBlockContext(object):

@ -18,6 +18,7 @@
#
# @end:license
from libc.stdint cimport uint8_t
from noisicaa.audioproc.public.musical_time cimport PyMusicalTime
from .buffer_arena cimport PyBufferArena
from . cimport message_queue
@ -70,6 +71,9 @@ cdef class PyBlockContext(object):
stime.start_time = start_time.get()
stime.end_time = end_time.get()
def set_input_events(self, uint8_t* buf):
self.__ctxt.input_events = <LV2_Atom_Sequence*>buf
@property
def perf(self):
return self.__perf

@ -214,6 +214,8 @@ Status Engine::loop(Realm* realm, Backend* backend) {
}
ctxt->perf->reset();
ctxt->input_events = nullptr;
RETURN_IF_ERROR(backend->begin_block(ctxt));
auto auto_end_block = scopeGuard([this, backend, ctxt]() {
Status status = backend->end_block(ctxt);
@ -228,12 +230,12 @@ Status Engine::loop(Realm* realm, Backend* backend) {
Buffer* buf = realm->get_buffer("sink:in:left");
if(buf != nullptr) {
RETURN_IF_ERROR(backend->output(ctxt, "left", buf->data()));
RETURN_IF_ERROR(backend->output(ctxt, Backend::Channel::AUDIO_LEFT, buf->data()));
}
buf = realm->get_buffer("sink:in:right");
if(buf != nullptr) {
RETURN_IF_ERROR(backend->output(ctxt, "right", buf->data()));
RETURN_IF_ERROR(backend->output(ctxt, Backend::Channel::AUDIO_RIGHT, buf->data()));
}
if (last_loop_time > chrono::high_resolution_clock::time_point::min()) {

@ -93,9 +93,7 @@ cdef class PyEngine(object):
self.__root_realm = None
self.__realm_listeners = {}
self.__backend = None
self.__backend_ready = threading.Event()
self.__backend_released = threading.Event()
self.__backend_listeners = {}
self.__engine_thread = None
self.__engine_started = None
@ -128,10 +126,7 @@ cdef class PyEngine(object):
self.__set_state(engine_notification_pb2.EngineStateChange.CLEANUP)
await self.stop_engine()
if self.__backend is not None:
self.__backend.cleanup()
self.__backend = None
self.stop_backend()
if self.__plugin_host is not None:
logger.info("Shutting down plugin host process...")
@ -160,6 +155,15 @@ cdef class PyEngine(object):
self.__set_state(engine_notification_pb2.EngineStateChange.STOPPED)
def stop_backend(self):
if self.__backend is not None:
self.__backend.cleanup()
self.__backend = None
for listener in self.__backend_listeners.values():
listener.remove()
self.__backend_listeners.clear()
async def start_engine(self):
assert self.__root_realm is not None
assert self.__backend is not None
@ -323,7 +327,6 @@ cdef class PyEngine(object):
if self.__backend is not None:
logger.info("Restarting backend...")
self.__backend.setup(self.__root_realm)
self.__backend_ready.set()
await self.start_engine()
@ -337,16 +340,15 @@ cdef class PyEngine(object):
self.__set_state(engine_notification_pb2.EngineStateChange.CLEANUP)
await self.stop_engine()
self.__backend.cleanup()
self.__backend = None
self.stop_backend()
self.__set_state(engine_notification_pb2.EngineStateChange.SETUP)
settings = PyBackendSettings(**parameters)
self.__backend = PyBackend(self.__host_system, name, settings)
self.__backend_listeners['notifications'] = self.__backend.notifications.add(
self.notifications.call)
self.__backend.setup(self.__root_realm)
self.__backend_ready.set()
logger.info("Backend '%s' ready.", name)

@ -482,7 +482,8 @@ class ChildRealmNode(Node):
spec.append_opcode(
'CALL_CHILD_REALM',
self.__child_realm,
self.outputs['out:left'].buf_name, self.outputs['out:right'].buf_name)
self.outputs['out:left'].buf_name,
self.outputs['out:right'].buf_name)
class Graph(object):

@ -258,6 +258,7 @@ Status run_CALL_CHILD_REALM(BlockContext* ctxt, ProgramState* state, const vecto
PerfStats* perf = realm->block_context()->perf.get();
perf->reset();
realm->block_context()->input_events = ctxt->input_events;
realm->block_context()->out_messages = ctxt->out_messages;
RETURN_IF_ERROR(realm->process_block(program));
realm->block_context()->out_messages = nullptr;

@ -381,7 +381,7 @@ Status Realm::set_spec(const Spec* s) {
return Status::Ok();
}
Buffer* Realm::get_buffer(const string& name) {
Buffer* Realm::get_buffer(const char* name) {
Program* program = _current_program.load();
if (program == nullptr) {
return nullptr;

@ -138,7 +138,7 @@ public:
Status run_maintenance();
StatusOr<BufferArena*> get_buffer_arena(uint32_t size);
Buffer* get_buffer(const string& name);
Buffer* get_buffer(const char* name);
private:
void activate_program(Program* program);

@ -57,7 +57,7 @@ cdef extern from "noisicaa/audioproc/engine/realm.h" namespace "noisicaa" nogil:
StatusOr[Program*] get_active_program()
Status process_block(Program* program)
Status run_maintenance()