B.Minor - 24 October 2014 11:17 AM

5pinDIN, thanks for pointing me to the right menu. It’s exactly as you assumed; right after the factory set, the sequencer memory was 295.6kB/896.0kB (33%), and after the “default” song/pattern bank store & restore procedure using Melas Tools the sequencer memory was completely filled up, displaying 896.0kB/896.0kB (100%).

What surprised me is the fact that the sequencer memory in the MOXF factory state (including 3 short demo songs and 3 simple patterns containing a few bars only) already uses 1/3 of the total available sequencer memory capacity. I guess it will never be possible to utilize all of the 64 pattern / 64 song allocations in a reasonable way.

[...]Yes, in the meantime I also followed this thread and I already performed the “clear song” and “clear pattern” procedures as described. [...] After a successful “clear” procedure the memory-meter will now really show 0kB/896.0kB (0%), and it will also stay like that - even if you switch the device off and on again!

Okay, after “clearing” the entire sequencer memory, I repeated the restore procedure from the application, sending back all “default” song/pattern data, again using Melas Tools. This time the banks where transferred over to the MOXF without any “Sequence memory full” error message.

But - surprise surprise - guess what the memory-meter displays this time: 704.0kB/896.0kB (79%). Howcome the value is still higher than the original value (704.0kB vs. 295.6kB), meaning that the required memory capacity has suddenly been more than doubled for holding the same data?

[...]Melas seems to administrate Mixing data only, no related track MIDI events (at least not in the MOXF version of the Total Librarian Tool). That means, only Mixing data for songs and patterns seem to be stored to / restored from the Total Librarian application. Therefore, if patterns and songs are manually “cleared” first on the MOXF before the Melas restore procedure takes place, also all included tracks and MIDI event data belonging to each pattern/song will be lost. Only the pure Mixing data will be restored back to the MOXF to each song and pattern, thus ending up with empty MIDI tracks in patterns and songs which don’t have a name and won’t “play” anymore.[...]

OK, I’ve got some time now to go into details…

It seems we agree that the Melas software deals only with Song/Pattern Mix data, not the sequenced MIDI events. Therefore, the increase from 295.6 kB to 704.0 kB can’t be attributed to MIDI event data.

While I don’t use Total Librarian, I have a demo copy of it, which I ran with my XF. I put the XF in Song mode, had the Melas program “Receive All...”, and watched the Song number on the XF’s display increment one-by-one from 01 to 64 while the Mix data was transmitted. Obviously, the program had requested the Mix data for each of the 64 Songs. The same was true for the Patterns. I had just previously done a factory reset, and the XF’s sequencer memory was storing only the factory sequences, five Songs and three Patterns. So there were only eight related Mixes - the other 120 were default Mix setups.

I next stored a total of 10 sequences to the XF’s memory from otherwise blank Songs and Patterns, so that only Mix data was actually being stored. I noted that the sequencer memory usage increased by 33 kB (the XF doesn’t provide any decimal places), which means that approximately 3.3 kB is used by the XF per stored Song or Pattern Mix. Checking the XF and MOXF Data Lists, the Mix data byte count is almost identical between the two models, so the MOXF likely also uses about 3.3 kB per stored Mix.

The 295.6 kB MOXF sequencer memory usage with the three factory Songs and three factory Patterns includes the MIDI event data plus Mix data for the six sequences. 128 less 6 is 122. 122 times 3.3 kB is 402.6 kB. From 295.6 kB to 704.0 kB, there’s an increase of 408.4 kB. Within a margin of error for the Mix data not being precisely 3.3 kB, I think I found the source of the increase in data size.

It seems Total Librarian doesn’t recognize when the Mix data is at the default settings. It therefore appears to assume that the data for each Song or Pattern Mix represents a Mix for an actual sequence, and considers all 128 as valid. When Total Librarian is requested to “Transmit All...”, it apparently sends all 128 Mixes back - the MOXF doesn’t know that it’s being sent unnecessary data, and then stores all 128 Mixes, not just the six representing actual sequences. Hence the increase in sequencer memory usage.

Perhaps a comparison could be made by Total Librarian of the data for each received Mix versus that of default Mix settings, and a note made if they matched. Then, when “Transmit All...” was selected, those mixes marked as matching the default would not be sent to the MOXF. That should prevent the unnecessary Mix data from being stored by the MOXF.

5pinDIN:
It seems Total Librarian doesn’t recognize when the Mix data is at the default settings. It therefore appears to assume that the data for each Song or Pattern Mix represents a Mix for an actual sequence, and considers all 128 as valid. When Total Librarian is requested to “Transmit All...”, it apparently sends all 128 Mixes back - the MOXF doesn’t know that it’s being sent unnecessary data, and then stores all 128 Mixes, not just the six representing actual sequences. Hence the increase in sequencer memory usage.

Okay, I see the point. Thanks a lot for your personal investigations, but I hope you can understand that I cannot be satisfied with this disappointing result.

EDIT:
In other words that would mean that the MOXF will just be able to store mix data for all of its 64 songs and 64 patterns, provided there are no MIDI track/event data available at all. However, there will not be enough space left to store even very small MIDI track/event data for all of those songs/patterns.

Sorry, but this is very frustrating to know. I have to agree with Jeff R 50 that I can’t understand what was in the designer’s minds when making such decisions. In this case the sequencer will be really useless for most cases.

Maybe at least the Melas Tools application can be adapted to skip those “default” mixes which will only cause “additional” memory usage in the MOXF when sent back, so that as much sequencer memory space as possible can be preserved for other useful sequencer data.

B.Minor - 30 October 2014 05:03 PM

5pinDIN:
It seems Total Librarian doesn’t recognize when the Mix data is at the default settings. It therefore appears to assume that the data for each Song or Pattern Mix represents a Mix for an actual sequence, and considers all 128 as valid. When Total Librarian is requested to “Transmit All...”, it apparently sends all 128 Mixes back - the MOXF doesn’t know that it’s being sent unnecessary data, and then stores all 128 Mixes, not just the six representing actual sequences. Hence the increase in sequencer memory usage.

Okay, I see the point. Thanks a lot for your personal investigations, but I hope you can understand that I cannot be satisfied with this disappointing result.

You’re welcome. I’m sorry that you think what I determined was a “disappointing result”. If I’m correct (remember, I’m 99.9% sure of it  :-) ), the problem you’re experiencing with the Melas software apparently isn’t the fault of the MOXF.

While I own and use several pieces of Yamaha gear, I’m not unaware or in denial of some of its limitations. However, when an issue comes up where the Yamaha product is being blamed for something that has an external cause, I’ll defend it.

 

B.Minor -

EDIT:
In other words that would mean that the MOXF will just be able to store mix data for all of its 64 songs and 64 patterns, provided there are no MIDI track/event data available at all. However, there will not be enough space left to store even very small MIDI track/event data for all of those songs/patterns.

I don’t know how you came to that conclusion. Even if all 64 Songs and all 64 Patterns had associated Mixes, that would use about 422 kB of the 896 kB total. The remaining 474 kB can hold quite a bit of MIDI event data, if used properly. Note-On or Note-Off messages only require three bytes. It’s usually the controller data that eats up large amounts of memory, if things are carefully managed.

 

B.Minor -

Sorry, but this is very frustrating to know. I have to agree with Jeff R 50 that I can’t understand what was in the designer’s minds when making such decisions. In this case the sequencer will be really useless for most cases.

I’m not going to try to defend the amount of sequencer memory in the MOXF, or any of the Motif line - but this seems to be a significant overstatement of the situation. The Motif sequencers are quite usable, especially if storage on a USB drive is taken advantage of.

 

B.Minor -

Maybe at least the Melas Tools application can be adapted to skip those “default” mixes which will only cause “additional” memory usage in the MOXF when sent back, so that as much sequencer memory space as possible can be preserved for other useful sequencer data.

Since loading sequencer data from a USB drive doesn’t cause the “additional” memory usage, it seems that would be the way to save it for now.

5pinDIN:
I don’t know how you came to that conclusion. Even if all 64 Songs and all 64 Patterns had associated Mixes, that would use about 422 kB of the 896 kB total. The remaining 474 kB can hold quite a bit of MIDI event data, if used properly.

I have to disagree on that. With my previous tests on the MOXF which I reported few posts ago, I could prove that after a total “clear” procedure on all patterns/songs (meaning no data were available anymore in the device which also showed 0kB) and sending back 128 pattern/song mixes using Melas Tools, already 704kB were occupied. 896kB available memory minus 704kB utilized mix memory leaves only 192kB left for any “real” MIDI sequencer data which is much too less to work with.

5PinDIN:
Since loading sequencer data from a USB drive doesn’t cause the “additional” memory usage, it seems that would be the way to save it for now.

Yes, as this seems to be the only way to store/save MIDI event data that’s the way to go. Anyway, I will ask John if the Total Librarian Tool can be adapted to treat “empty” instances differently which could bring further improvements for all Motif series.

Jeff R 50 - 29 October 2014 10:18 PM

5pin, it shows I have used up 859.5 KB of the included 896.0 KB available. I have only three - 5 minute songs recorded on the keyboard, and each has only about 5 tracks used.  Seriously, I think my electric razer has more memory than this keyboard. I’m stunned that there is actually less than 1 MB available in this board.

Lets’ try some simple math. I’m going to pick numbers that make the calculations easy.

Assume a Song of average tempo, 120 bpm. 120 bpm = 2 beats per second.

Assume 5 notes per beat for each and every Part/Voice (that’s quite a few for some instruments). With 5 “tracks”, that would yield 50 notes/sec.

Three 5 minute Songs = 15 minutes = 900 seconds. In 900 seconds, with the previous assumptions that’s 45,000 notes.

MOXF note capacity is approx 226,000. 45,000 is about 1/5 of that capacity.

If the sequencer memory is nearly full with data from only your five Songs, here are a couple of possible causes:
1) There may be a heck of a lot of controller data. See your previous thread http://www.motifator.com/index.php/forum/showflat/viewthread/471439/
2) Perhaps more than your three Songs are in the memory. You could first save your Songs to a USB drive. Then clear all Songs and Patterns from the MOXF ("all" is the setting for “Clear” immediately after “64"), and verify that the sequencer memory usage is indicated as 0 kB. Then load your Songs back in. If they still take up nearly all the sequencer memory, you may need to reduce controller data.

B.Minor - 30 October 2014 07:42 PM

5pinDIN:
I don’t know how you came to that conclusion. Even if all 64 Songs and all 64 Patterns had associated Mixes, that would use about 422 kB of the 896 kB total. The remaining 474 kB can hold quite a bit of MIDI event data, if used properly.

I have to disagree on that. With my previous tests on the MOXF which I reported few posts ago, I could prove that after a total “clear” procedure on all patterns/songs (meaning no data were available anymore in the device which also showed 0kB) and sending back 128 pattern/song mixes using Melas Tools, already 704kB were occupied. 896kB available memory minus 704kB utilized mix memory leaves only 192kB left for any “real” MIDI sequencer data which is much too less to work with.

Would you please recheck that? 704 kB for only Mix data seems excessive, even for all 128 possible Mixes.

EDIT: I need to correct something I wrote earlier. I didn’t realize that Song and Pattern Mix data are of different length. I just did another check, first with 10 Song Mixes, and then with 10 Pattern Mixes. It seems that Song mixes are about 3 kB, while Pattern Mixes are about 5 kB. If all 64 of both Song and Pattern Mixes were stored, that would total about 512 kB, at least for an XF. That still leaves me wondering what could total 704 kB - I know the sequencers in the XF and MOXF are different, but I’m not sure that accounts for an additional 192 kB.

5pinDIN:
Would you please recheck that? 704 kB for only Mix data seems excessive, even for all 128 possible Mixes.

Unfortunately I have to confirm my previous test results for the MOXF. Even though I already checked the results twice before I initially posted them, I did one more test. This time I also included all intermediate results by checking the “memory-meter” after each step. I guess this will provide more insight, especially which action releases or consumes how much MOXF sequencer memory. Here’s what I did:

--------------------------------------------------------------------------------------------------------
-> performing a factory set on MOXF
-> sending all song/pattern mixes to Total Librarian
-> initializing “clear” procedures on the MOXF performing the following steps:
--------------------------------------------------------------------------------------------------------
295.6kB: right after MOXF factory set
118.8kB: right after manually “clearing” all default songs (176.8kB were released)
0.0kB: right after manually “clearing” all default patterns/chains in addition (118.8kB were released)
--------------------------------------------------------------------------------------------------------
-> checking that MOXF sequencer memory really remains unoccupied
-> transmitting back song and pattern mixes from Total Librarian to MOXF in the following steps:
--------------------------------------------------------------------------------------------------------
0.0kB: right before transmitting any mix data to the MOXF
304.0kB: right after transmitting mix data for all 64 songs (304.0kB were occupied)
704.0kB: right after transmitting mix data for all 64 patterns in addition (400kB were occupied)
--------------------------------------------------------------------------------------------------------

So, in fact for the MOXF it turns out that each song mix will utilize 4.75kB (304kB/64) and each pattern mix will utilize 6.25kB (400kB/64). This finally leaves only 192kB (896kB-704kB) which will be available for any MIDI events on the MOXF - meaning for all patterns and songs. Under normal circumstances you can only record one “average” song with this kind of memory amount. That was also the reason why I called this (for me unacceptable) result “disappointing”.

5pinDIN:
EDIT: I need to correct something I wrote earlier. I didn’t realize that Song and Pattern Mix data are of different length. I just did another check, first with 10 Song Mixes, and then with 10 Pattern Mixes. It seems that Song mixes are about 3 kB, while Pattern Mixes are about 5 kB. If all 64 of both Song and Pattern Mixes were stored, that would total about 512 kB, at least for an XF. That still leaves me wondering what could total 704 kB - I know the sequencers in the XF and MOXF are different, but I’m not sure that accounts for an additional 192 kB.

That’s why I was surprised, too. I’m also wondering why there are such big differences to the results you get after doing the same Melas tests on your Motif XF. However, I hope you are now convinced that there must be something wrong with the sequencer memory allocation in the MOXF. Why should any sequencer mix data consume more memory than similar data on the Motif, thus not leaving enough headroom for event data, even tightened by the fact that there’s a smaller sequencer memory capacity available in the MOXF?

B.Minor - 31 October 2014 03:54 AM

So, in fact for the MOXF it turns out that each song mix will utilize 4.75kB (304kB/64) and each pattern mix will utilize 6.25kB (400kB/64). This finally leaves only 192kB (896kB-704kB) which will be available for any MIDI events on the MOXF - meaning for all patterns and songs. Under normal circumstances you can only record one “average” song with this kind of memory amount. That was also the reason why I called this (for me unacceptable) result “disappointing”.

You’re apparently basing your disappointment on the result of using Total Librarian, assuming simultaneous storage of Mixes for all 128 MOXF Songs and Patterns is “normal”.  However, there’s a noticeable flaw in that…
Your premise is that you’ll have Mixes for all 128 Songs/Patterns, but then only record one “average” Song. That type of usage isn’t at all typical - in fact, it’s so unlikely that I doubt anyone would use the sequencer that way. Typical operation is to record some Songs and/or Patterns, and have Mixes related to those sequences. The hundreds of kB of unnecessary Mix data just wouldn’t exist with normal usage, leaving significant capacity for “average” Songs/Patterns.

Let’s hope that John Melas can resolve the situation, but in the meantime focusing on the MOXF’s behavior separately might be more helpful in determining whether there are definable problems with its sequencer.

 

B.Minor -

5pinDIN:
EDIT: I need to correct something I wrote earlier. I didn’t realize that Song and Pattern Mix data are of different length. I just did another check, first with 10 Song Mixes, and then with 10 Pattern Mixes. It seems that Song mixes are about 3 kB, while Pattern Mixes are about 5 kB. If all 64 of both Song and Pattern Mixes were stored, that would total about 512 kB, at least for an XF. That still leaves me wondering what could total 704 kB - I know the sequencers in the XF and MOXF are different, but I’m not sure that accounts for an additional 192 kB.

That’s why I was surprised, too. I’m also wondering why there are such big differences to the results you get after doing the same Melas tests on your Motif XF. However, I hope you are now convinced that there must be something wrong with the sequencer memory allocation in the MOXF. Why should any sequencer mix data consume more memory than similar data on the Motif, thus not leaving enough headroom for event data, even tightened by the fact that there’s a smaller sequencer memory capacity available in the MOXF?

The demo version of Total Librarian I have is crippled from sending data back to the XF. My determination of the size of Mix data was done on the XF. I made note of the current sequencer memory usage, then stored Songs having no actual MIDI event data to 10 empty locations, noted the increase in memory usage, and divided by 10. I then did the same for Patterns. That’s how I determined that the XF uses about 3 kB per Song Mix and 5 kB per Pattern Mix. Would you try that approach on your MOXF, and see how it compares to the XF and to the 4.75kB/6.25kB determinations made as the result of using Total Librarian?

5pinDIN:
You’re apparently basing your disappointment on the result of using Total Librarian, assuming simultaneous storage of Mixes for all 128 MOXF Songs and Patterns is “normal”.  However, there’s a noticeable flaw in that…
Your premise is that you’ll have Mixes for all 128 Songs/Patterns, but then only record one “average” Song. That type of usage isn’t at all typical - in fact, it’s so unlikely that I doubt anyone would use the sequencer that way.
Typical operation is to record some Songs and/or Patterns, and have Mixes related to those sequences. The hundreds of kB of unnecessary Mix data just wouldn’t exist with normal usage, leaving significant capacity for “average” Songs/Patterns.

5pinDIN, I could also interpret some of your statements as a nice try to sweep the genuine problem under the carpet. Of course it is not a “used case” to fill up all 128 mixes with any “dummy” data first, just to have a good reason to complain afterwards about the fact that only sequencer memory for one single song will be left. However, this is just an example how the MOXF is currently managing its internal sequencer data, and in fact that example cannot even be considered as a worst case. However, after all you also couldn’t believe at first that already 79% (704kB) of the whole sequencer memory capacity is already occupied by pure mixing data, not even having stored one single MIDI event in the MOXF sequencer memory, right?

5pinDIN:
Would you please recheck that? 704 kB for only Mix data seems excessive, even for all 128 possible Mixes.

But exactly that situation illustrates that there has been a conceptual flaw in the memory design, namely by not considering the situation that whole song/pattern mixes might be sent back and forth as whole banks between external applications/devices and the MOXF. It doesn’t matter if these applications are sequencers or editors like Melas.

Since dropping in here in this thread I made very clear that my problem is more SysEx related, not based on procedures how to reduce MIDI event data by not using certain controllers, or how to store/restore frequently to/from USB. Apparently nobody wants to talk about those SysEx messages; instead I’ve been pushed to use conventional methods only to manage my libraries. I wouldn’t have bought such a comprehensive tool like Total Librarian if I had planned to manage every setting manually only. After all the whole bunch of SysEx messages have been implemented by Yamaha, especially for the purpose of communicating with external instances by exchanging parameters and settings, so synthesizer users should be convinced that such options would also work out the same way as setting parameters manually, but not ending up in too less available memory without any manual intervention. The user can’t be blamed for the fact that the MOXF currently isn’t smart enough to decide by itself which data have to be internally saved in its memory and which not.

It’s clear that MIDI event based sequencer memory capacity cannot be endless. But in terms of mix parameters for each song/pattern (which takes much less space compared to MIDI tracks), the design could have also been similar to voices/performances/masters where always a certain amount of space for the essential content is reserved, regardless if the locations are currently empty/initialized, they contain garbage or even useful data. As the sequencer memory space cannot be managed by useres in terms of “clearing” MIDI event data or mixes separately (just all at once), how could MOXF owners be blamed for the fact that a “Sequence memory full” error can occur in that particular case? Again, if 128 SysEx instructions for transmitting each song/pattern mix can be actively triggered by the MOXF in order to send it to any external application for archive (regardless if these songs/patterns are actually empty or not), every user could expect that the same 128 mix data can also be passively received back again from the MOXF without experiencing any memory overflow.

5pinDIN:
The demo version of Total Librarian I have is crippled from sending data back to the XF. My determination of the size of Mix data was done on the XF. I made note of the current sequencer memory usage, then stored Songs having no actual MIDI event data to 10 empty locations, noted the increase in memory usage, and divided by 10. I then did the same for Patterns. That’s how I determined that the XF uses about 3 kB per Song Mix and 5 kB per Pattern Mix.
Would you try that approach on your MOXF, and see how it compares to the XF and to the 4.75kB/6.25kB determinations made as the result of using Total Librarian?

Of course I would, but first I’d like to understand what you exactly did while performing your tests. In particular I don’t get the phrase…

[...] then stored Songs having no actual MIDI event data to 10 empty locations [...]

What’s exactly the procedure to create/generate patterns/songs in the MOXF with no MIDI event data inside but having mix data applied to them? Can this be done from scratch by just selecting “empty” songs/patterns and storing them to another position? But how will they ever get any mix data that way? Isn’t the Total Librarian application actually performing this procedure, namely also applying mix data to “empty” songs/patterns? Would it really change the memory occupation per song/pattern mix data set if I only test 10 patterns/songs out of 128? Sorry, but I don’t get what you want me to do. Maybe you can explain again how you set up your memory tests on the Motif XF.

5pinDIN:
Would you try that approach on your MOXF, and see how it compares to the XF and to the 4.75kB/6.25kB determinations made as the result of using Total Librarian?

Okay, I think I understood now what is your goal. I did another test, this time without using Melas Tools at all, just using the internal capabilities in the MOXF itself.

At first I performed a factory set, followed by a complete “clear” procedure for all songs and patterns. The result was 0kB sequencer memory in use - as expected.

Then I went into song mode, opened the first “empty” song #1, pressed the Mixing button, changed only the first parameter from “C” to 63 (PAN setting) and stored the song back again to its original position. I repeated the same for the next four “empty” songs, namely for #2 to #5.

After that I changed to pattern mode and I performed the same procedure again in conjunction with the Mixing button, by changing the first mixing parameter only again, for each of the first five “empty” patterns #1 to #5 and storing them back to their original positions.

In fact, in total I ended up in having “generated” 10 sequences with no MIDI data included, but having mixing parameter sections inside. Here’s what the memory displayed after each step:

-> after the initial memory set followed by the song/pattern “clear” procedures: 0kB

-> after storing the first 5 songs including their mixing parameter sets (only 1 single parameter changed per song):
23.7kB

-> after additionally storing the first 5 patterns including their mixing parameter sets (only 1 single parameter changed per pattern):
55.0kB (meaning an additional increase of 31.3kB)

Comparing these results with the ones I got from my previous test when using Melas Tools, it seems to me that they are almost equal (considering that there may be a small rounding error behind the decimal point when displayed in the MOXF display):

MOXF Songs: 23.7kB/5 = 4.74kB per song
MOXF Patterns: 31.3kB/5 = 6,26 per pattern

So again, also Melas Tools can’t be blamed for that high amount of sequencer data usage for mix data. It doesn’t matter which method or tool you choose for setting mixing parameters. Of course this still doesn’t explain why the MOXF is requiring much more sequencer memory space than the Motif XF does for the same kind of sequencer data. Any ideas?

But let me also mention the most important finding which seems to be the key here:

If the procedure mentioned above is just performed without changing any mix parameters (entering the mixing parameters, but leaving all of them where they are) and storing the song/pattern back, no additional data will be added. The “empty” song or pattern will really stay unaltered. So in fact the MOXF seems to be capable of checking the content before storage, especially if any of its mix parameters are deviating from “default” or not.

In contrast to that, if the same “empty"/unaltered song/pattern is sent to the MOXF back via SysEx, e.g. by the Total Librarian application, additional memory will automatically be occupied inside the MOXF in order to store these “default” (= unnecessary) values back to memory. That’s exactly the point were I mentioned before that it would be required to make the MOXF smart enough to know if such a storage is necessary at all.

B.Minor - 31 October 2014 03:47 PM

5pinDIN, I could also interpret some of your statements as a nice try to sweep the genuine problem under the carpet. Of course it is not a “used case” to fill up all 128 mixes with any “dummy” data first, just to have a good reason to complain afterwards about the fact that only sequencer memory for one single song will be left. However, this is just an example how the MOXF is currently managing its internal sequencer data, and in fact that example cannot even be considered as a worst case. However, after all you also couldn’t believe at first that already 79% (704kB) of the whole sequencer memory capacity is already occupied by pure mixing data, not even having stored one single MIDI event in the MOXF sequencer memory, right?

I’m not such a devoted Yamaha fan that I refuse to see flaws in their products. I really do want to help determine what’s going on here. It’s possible you’re unaware of this, since you might not frequent the XS or XF forums, but I have pointed out bugs in the XS and XF operation. (By the way, those faults have never been resolved from one OS version to the next.) So no, I’m not trying to “sweep the genuine problem under the carpet”, just attempting to determine where the root of the problem lies.

 

B.Minor - 31 October 2014 03:47 PM

5pinDIN:
Would you please recheck that? 704 kB for only Mix data seems excessive, even for all 128 possible Mixes.

But exactly that situation illustrates that there has been a conceptual flaw in the memory design, namely by not considering the situation that whole song/pattern mixes might be sent back and forth as whole banks between external applications/devices and the MOXF. It doesn’t matter if these applications are sequencers or editors like Melas.

Since dropping in here in this thread I made very clear that my problem is more SysEx related, not based on procedures how to reduce MIDI event data by not using certain controllers, or how to store/restore frequently to/from USB. Apparently nobody wants to talk about those SysEx messages; instead I’ve been pushed to use conventional methods only to manage my libraries.
[...]
Again, if 128 SysEx instructions for transmitting each song/pattern mix can be actively triggered by the MOXF in order to send it to any external application for archive (regardless if these songs/patterns are actually empty or not), every user could expect that the same 128 mix data can also be passively received back again from the MOXF without experiencing any memory overflow.

It’s not that I don’t want to talk about SysEx. I’ve written software for synths, including an edlib (editor/librarian) for the Yamaha YS200 back in the good old days of DOS. That, of course, relied on SysEx, and I have no problem interpreting Data Lists, MIDI Implementation Charts, etc. In the process I found that sometimes a SysEx message didn’t do what the manufacturer claimed, and that even when it did could sometimes have unanticipated consequences.

May I suggest that you contact John Melas, and ask him which single SysEx message Total Librarian sends to the MOXF to request the Mix data for all 64 Songs and Patterns? Also please ask him how Total Librarian sends a “whole bank” of Mixes back to the MOXF. You might find his reply enlightening, if it’s forthcoming - and if so, please share it with us.

Once the Melas software requests the Mix for each and every Song and/or Pattern, sending all that Mix data back to the MOXF gives it legitimacy. Since Mix data is “divorced” from Song/Pattern MIDI events, the MOXF has no way of knowing whether what it’s being sent is a “real” Mix or not. (You could have recorded a Piano piece without changing the default Mix setup.) The MOXF has no choice but to accept a Mix it’s sent as being “real”, and if you send it 128 of them, it’s going to use the sequencer memory necessary to store them.

 

B.Minor -

5pinDIN:
The demo version of Total Librarian I have is crippled from sending data back to the XF. My determination of the size of Mix data was done on the XF. I made note of the current sequencer memory usage, then stored Songs having no actual MIDI event data to 10 empty locations, noted the increase in memory usage, and divided by 10. I then did the same for Patterns. That’s how I determined that the XF uses about 3 kB per Song Mix and 5 kB per Pattern Mix.
Would you try that approach on your MOXF, and see how it compares to the XF and to the 4.75kB/6.25kB determinations made as the result of using Total Librarian?

Of course I would, but first I’d like to understand what you exactly did while performing your tests. In particular I don’t get the phrase…

[...] then stored Songs having no actual MIDI event data to 10 empty locations [...]

What’s exactly the procedure to create/generate patterns/songs in the MOXF with no MIDI event data inside but having mix data applied to them? Can this be done from scratch by just selecting “empty” songs/patterns and storing them to another position? But how will they ever get any mix data that way? Isn’t the Total Librarian application actually performing this procedure, namely also applying mix data to “empty” songs/patterns? Would it really change the memory occupation per song/pattern mix data set if I only test 10 patterns/songs out of 128? Sorry, but I don’t get what you want me to do. Maybe you can explain again how you set up your memory tests on the Motif XF.

Yes, just select “empty” Songs/Patterns and store them to a previously unused location. Every time you store one the memory usage will increase by a few kB (3 or 5 kB on the XF). I did ten Songs, then divided by ten, potentially to increase accuracy. I then did the same with ten Patterns.

5pinDIN:
Yes, just select “empty” Songs/Patterns and store them to a previously unused location. Every time you store one the memory usage will increase by a few kB (3 or 5 kB on the XF). I did ten Songs, then divided by ten, potentially to increase accuracy. I then did the same with ten Patterns.

Please refer to my findings posted before your last reply.

Edit (added):

5pinDIN:
May I suggest that you contact John Melas, and ask him which single SysEx message Total Librarian sends to the MOXF to request the Mix data for all 64 Songs and Patterns? Also please ask him how Total Librarian sends a “whole bank” of Mixes back to the MOXF. You might find his reply enlightening, if it’s forthcoming - and if so, please share it with us.

I never said that there is a single instruction for sending whole pattern/song banks - just the treatment as such. I even pointed that explicitely out in the following chapter of the same post you’re referring to:

B.Minor:
Again, if 128 SysEx instructions for transmitting each song/pattern mix can be actively triggered by the MOXF in order to send it to any external application for archive (regardless if these songs/patterns are actually empty or not), every user could expect that the same 128 mix data can also be passively received back again from the MOXF without experiencing any memory overflow.

Another example from one of my very early posts to this topic:

B.Minor:
[...] In fact it seems to be the same procedure as sending each pattern/song one after another manually to the unit, but only as a scripted process.

B.Minor - 31 October 2014 06:32 PM

5pinDIN:
Would you try that approach on your MOXF, and see how it compares to the XF and to the 4.75kB/6.25kB determinations made as the result of using Total Librarian?

[...]
In fact, in total I ended up in having “generated” 10 sequences with no MIDI data included, but having mixing parameter sections inside. Here’s what the memory displayed after each step:

-> after the initial memory set followed by the song/pattern “clear” procedures: 0kB

-> after storing the first 5 songs including their mixing parameter sets (only 1 single parameter changed per song):
23.7kB

-> after additionally storing the first 5 patterns including their mixing parameter sets (only 1 single parameter changed per pattern):
55.0kB (meaning an additional increase of 31.3kB)

Comparing these results with the ones I got from my previous test when using Melas Tools, it seems to me that they are almost equal (considering that there may be a small rounding error behind the decimal point when displayed in the MOXF display):

MOXF Songs: 23.7kB/5 = 4.74kB per song
MOXF Patterns: 31.3kB/5 = 6,26 per pattern

OK, that certainly appears to agree with your previous determination.

 

B.Minor -

So again, also Melas Tools can’t be blamed for that high amount of sequencer data usage for mix data. It doesn’t matter which method or tool you choose for setting mixing parameters.

It seems we’ll have to agree to disagree on that. While Total Librarian isn’t responsible for the amount of memory used for each stored Mix, sending Mix data to the MOXF for otherwise “empty” Songs and Patterns doesn’t make sense to me.

 

B.Minor -

Of course this still doesn’t explain why the MOXF is requiring much more sequencer memory space than the Motif XF does for the same kind of sequencer data. Any ideas?

I mentioned before that the sequencers in the XF and MOXF differ. I don’t know the specifics, or how that might affect memory usage for Mixes.

 

B.Minor -

But let me also mention the most important finding which seems to be the key here:

If the procedure mentioned above is just performed without changing any mix parameters (entering the mixing parameters, but leaving all of them where they are) and storing the song/pattern back, no additional data will be added. The “empty” song or pattern will really stay unaltered.

Interesting - that’s not the way the XF functions. Storing an unaltered/default Mix results in additional sequencer memory usage.

 

B.Minor -

So in fact the MOXF seems to be capable of checking the content before storage, especially if any of its mix parameters are deviating from “default” or not.

I think it’s unlikely that the MOXF compares the content to default values. I suspect it just detects that no edit has been done. You could determine if that’s the case by changing one parameter as you did previously, then changing it back to what it was initially, and finally storing the Song/Pattern. If you try that, please let us know what you determine.

 

B.Minor -

In contrast to that, if the same “empty"/unaltered song/pattern is sent to the MOXF back via SysEx, e.g. by the Total Librarian application, additional memory will automatically be occupied inside the MOXF in order to store these “default” (= unnecessary) values back to memory. That’s exactly the point were I mentioned before that it would be required to make the MOXF smart enough to know if such a storage is necessary at all.

I’m not ready to accept that there is a “contrast” - see my paragraph above.

IMO, it’s the job of the software to determine if it’s reading default Mixes, not that of the MOXF to reject those sent to it. In the long run, Mr. Melas and Yamaha can decide whose responsibility it is, and what should be done. Hopefully they won’t just point fingers at each other. I’ve already mentioned a possible solution:
“Perhaps a comparison could be made by Total Librarian of the data for each received Mix versus that of default Mix settings, and a note made if they matched. Then, when “Transmit All...” was selected, those mixes marked as matching the default would not be sent to the MOXF. That should prevent the unnecessary Mix data from being stored by the MOXF.”

B.Minor - 31 October 2014 07:10 PM

5pinDIN:
Yes, just select “empty” Songs/Patterns and store them to a previously unused location. Every time you store one the memory usage will increase by a few kB (3 or 5 kB on the XF). I did ten Songs, then divided by ten, potentially to increase accuracy. I then did the same with ten Patterns.

Please refer to my findings posted before your last reply.

Thanks - referred to and replied to.  :-)

 

B.Minor -

Edit (added):

5pinDIN:
May I suggest that you contact John Melas, and ask him which single SysEx message Total Librarian sends to the MOXF to request the Mix data for all 64 Songs and Patterns? Also please ask him how Total Librarian sends a “whole bank” of Mixes back to the MOXF. You might find his reply enlightening, if it’s forthcoming - and if so, please share it with us.

I never said that there is a single instruction for sending whole pattern/song banks - just the treatment as such. I even pointed that explicitely out in the following chapter of the same post you’re referring to:

B.Minor:
Again, if 128 SysEx instructions for transmitting each song/pattern mix can be actively triggered by the MOXF in order to send it to any external application for archive (regardless if these songs/patterns are actually empty or not), every user could expect that the same 128 mix data can also be passively received back again from the MOXF without experiencing any memory overflow.

Another example from one of my very early posts to this topic:

B.Minor:
[...] In fact it seems to be the same procedure as sending each pattern/song one after another manually to the unit, but only as a scripted process.

And I never said that you said there were single messages for requesting all the Mix data or sending whole banks. I was hoping to have John Melas participate in this, and that such questions might encourage him to become interested and engaged.

Since I don’t own a MOXF, my own interest in this topic is waning.  :-)

5pinDIN:
I think it’s unlikely that the MOXF compares the content to default values. I suspect it just detects that no edit has been done. You could determine if that’s the case by changing one parameter as you did previously, then changing it back to what it was initially, and finally storing the Song/Pattern. If you try that, please let us know what you determine.

In the meantime I also got curious to find out what’s the final trigger to make the MOXF assign a correlated “mix data memory area” to an “empty” song or pattern, and if it’s based on any “default” content checks or just if it detects that some content has been altered/edited.

As already mentioned earlier and checked again, the MOXF will not assign any additional mix parameter memory area (about 4.75kB for each song / 6.25kB for each pattern) after being stored back, if the content has also not been altered by the user, not even if the Mixing mode was entered and left again before storage (damn - I wish I could express myself better if English).

But if you believe it or not, you can repeat the procedure by entering the Mixing section, change a single parameter (so that the “E” sign will come up to indicate internal changes), set this parameter then back to its initial “default” value (the “E” is still there) and store the pattern/song back. The stunning thing now is that in this case also no additional mix memory area is assigned by the MOXF; in fact there will be no
additional memory usage for “edited default” mix data.

In contrast to that - as I showed in the other tests before - if you only change a single mix parameter and store the song/pattern back, the whole mix parameter memory area will already be assigned for that particular song/pattern, thus lowering the available sequencer memory space. However, if you then try to “re-edit” the song/pattern mix parameter back to its “default” value and store the song/pattern another time, of course the allocated mix parameter memory will not be released again for free usage - but that is something nobody would have expected anyway ;-).

So, however you might interpret these findings, there seems to be a kind of “intelligent” content checking before unneccesary memory is wasted in the MOXF.

5pinDIN:
IMO, it’s the job of the software to determine if it’s reading default Mixes, not that of the MOXF to reject those sent to it. In the long run, Mr. Melas and Yamaha can decide whose responsibility it is, and what should be done.
Hopefully they won’t just point fingers at each other. I’ve already mentioned a possible solution:
“Perhaps a comparison could be made by Total Librarian of the data for each received Mix versus that of default Mix settings, and a note made if they matched. Then, when “Transmit All...” was selected, those mixes marked as matching the default would not be sent to the MOXF. That should prevent the unnecessary Mix data from being stored by the MOXF.”

I had to smile a little bit when I read how you were phrasing your proposed solution for Melas, as it almost matches what I’ve been writing to John on Thursday night - the same proposal, but just in other words:

“I’m not a programmer, but maybe it’s possible to have a “flag” set when “Receiving all...” (or just when receiving a single item) by the Total Librarian library whenever a pattern or song hasn’t been altered from its default/initialized state. Whenever such a “default” pattern or song (or even a whole “bank") is then selected to be sent back to the device, maybe the application can check the contents first in order to exclude/skip all “flagged” items to prevent the MOXF from allocating any “additional” sequencer memory for them in the MOXF.”

I hope that John is responding soon and he can already tell me if he is willing to do that change in his next update for the Total Librarian application. This could be an improvement for all Motif series.

However, as a MOXF user I’m also expecting something back from the Yamaha guys. Or maybe a contribution from other persons who could help clarifying why the amount of required sequencer memory for mix data in the MOXF has to be much higher than for the same data in the XF, thus lowering the possibilities for MOXF users. If this amount could be lowered by design, I’m sure that this “Sequence memory full” error would occur less often.