-
Audio to MIDI conversion through audio recognition techniques. User commands service through speech recognition techniques. Automatic audio / speech discrimination.
This is a problem of the integration of two different technologies and to define mechanisms to assess the context: command or sound. In addition, the audio to midi has been done on an instrument considering innovation and didactical aspects.
-
User performance level evaluation.
This is a quite new and open problem, metric definition and teacher validation of them.
-
Musical exercises / courses organisation.
A series of musical exercises / courses consisting of suitable musical pieces has been stored, in forms of digitised score images, MIDI and wave files, in a multimedia database and used as patterns of reference for both objective (MIDI) and subjective (acoustic) comparison and evaluation.
-
Musical score processing.
The system allows the user to import musical pieces from written music scores through optical music recognition and conversion to MIDI format. This allows for maximum flexibility through customisation of course content. Present tools are quite simple, they permit the conversion of quite simple music therefore a more integrated tool was needed for the educational purpose.
Another useful tool is a graphical music editor of MIDI files. This will provide the means to edit the score of a musical piece, to restore it in MIDI format and hear the transformed score. In this way the user has the capability to monitor acoustically any changes (in frequencies or duration times) he makes to his MIDI output. Thus he can obtain a direct realization of all the musical phenomena through an interaction with the system. The editor is capable of generating MIDI outputs considering classical symbols plus ornaments (turns, trill, etc.), and the most common accents.
The above means that the IMUTUS project has a music notation editor integrated with a midi editor in terms of signal and that the production of MIDI is capable of reproducing ornaments, etc. that are presently non considered and nor reproduced in other tools such as FINALE; SIBELIUS; etc. only some simple example with TUrns have been provided in some tools, that are poor from other aspects (see Master Play along distributed by Schoot)
-
Virtual reality support.
The teaching process is enhanced through a virtual reality module. The virtual teacher appears during the playing of the original score or the recorded piece. The student, paying attention to the sequence of the virtual teacher fingerings synchronously with music, participates to a more realistic lesson. The adoption of VR allows to (i) show the hand movements on the basis of the MIDI input, (ii) show different view points, etc. In this content new pedagogical mechanisms have been defined and validated.
This is a quite new area, where many different aspects had to be covered. On the market there are some rigid examples that provide animation and 3d animations produced specifically for the example provided. No flexibility. The intention was to define and build a tool capable of reproducing the hand position and movements on the basis of the music that could be written even by the student.
-
Multimedia music tuition courseware. A full tuition course concerning general issues about music and topics about the selected instrument will be incorporated in the system. The course has been enriched by multimedia assets (audio, video, still images), giving a user-friendly view and functionality to the whole environment.
-
User friendly front-end environment. The operation environment of the user, where all the modules have been positioned, has been designed with guiding criteria the simplicity, the user friendliness and the functional efficiency. The usability of the remote tuition systems has a strong impact on the pedagogical aspects and quality of the distance teaching and learning. These aspects have been taken into account in order to reduce the gap and to better adapt the old techniques to the new solutions.
-
Validation and assessment of the system. The system has been validated and tested in realistic settings in order to verify the effective value of the model and solutions that have been proposed in the research phases of the project towards adoption of the model in real environments. Also this aspect is very important since most of the commercial products are not accepted in the music schools since they are too far from the real needs.