B.Minor - 01 November 2014 04:32 AM

5pinDIN:
I think it’s unlikely that the MOXF compares the content to default values. I suspect it just detects that no edit has been done. You could determine if that’s the case by changing one parameter as you did previously, then changing it back to what it was initially, and finally storing the Song/Pattern. If you try that, please let us know what you determine.

In the meantime I also got curious to find out what’s the final trigger to make the MOXF assign a correlated “mix data memory area” to an “empty” song or pattern, and if it’s based on any “default” content checks or just if it detects that some content has been altered/edited.

As already mentioned earlier and checked again, the MOXF will not assign any additional mix parameter memory area (about 4.75kB for each song / 6.25kB for each pattern) after being stored back, if the content has also not been altered by the user, not even if the Mixing mode was entered and left again before storage (damn - I wish I could express myself better if English).

But if you believe it or not, you can repeat the procedure by entering the Mixing section, change a single parameter (so that the “E” sign will come up to indicate internal changes), set this parameter then back to its initial “default” value (the “E” is still there) and store the pattern/song back. The stunning thing now is that in this case also no additional mix memory area is assigned by the MOXF; in fact there will be no
additional memory usage for “edited default” mix data.

In contrast to that - as I showed in the other tests before - if you only change a single mix parameter and store the song/pattern back, the whole mix parameter memory area will already be assigned for that particular song/pattern, thus lowering the available sequencer memory space. However, if you then try to “re-edit” the song/pattern mix parameter back to its “default” value and store the song/pattern another time, of course the allocated mix parameter memory will not be released again for free usage - but that is something nobody would have expected anyway ;-).

So, however you might interpret these findings, there seems to be a kind of “intelligent” content checking before unneccesary memory is wasted in the MOXF.

Based on your findings, the MOXF behavior differs from that of the XF in several ways.

It might be interesting to determine the size of bulk dumps of Song and Pattern Mixes from the MOXF (see page 121 of the Reference Manual). With the XF, either of them is 2903 bytes. That’s in contrast to the 3 kB sequencer memory used by the XF for a Song Mix, and 5 kB for a Pattern Mix. (I used MIDI-OX to capture the dumps. http://www.midiox.com/) For a Song, the memory usage is about the size of the bulk data, but it’s significantly more for a Pattern. I don’t know what accounts for the additional Pattern Mix data memory usage, given that the bulk data is sized the same as for a Song (with the XF, anyway).

By the way, your English is good - a lot better than my German. Mein Deutsch ist sehr schlecht. “Deutsche Sprache, schwere Sprache." ;-)

 

B.Minor -

5pinDIN:
IMO, it’s the job of the software to determine if it’s reading default Mixes, not that of the MOXF to reject those sent to it. In the long run, Mr. Melas and Yamaha can decide whose responsibility it is, and what should be done.
Hopefully they won’t just point fingers at each other. I’ve already mentioned a possible solution:
“Perhaps a comparison could be made by Total Librarian of the data for each received Mix versus that of default Mix settings, and a note made if they matched. Then, when “Transmit All...” was selected, those mixes marked as matching the default would not be sent to the MOXF. That should prevent the unnecessary Mix data from being stored by the MOXF.”

I had to smile a little bit when I read how you were phrasing your proposed solution for Melas, as it almost matches what I’ve been writing to John on Thursday night - the same proposal, but just in other words:

“I’m not a programmer, but maybe it’s possible to have a “flag” set when “Receiving all...” (or just when receiving a single item) by the Total Librarian library whenever a pattern or song hasn’t been altered from its default/initialized state. Whenever such a “default” pattern or song (or even a whole “bank") is then selected to be sent back to the device, maybe the application can check the contents first in order to exclude/skip all “flagged” items to prevent the MOXF from allocating any “additional” sequencer memory for them in the MOXF.”

I hope that John is responding soon and he can already tell me if he is willing to do that change in his next update for the Total Librarian application. This could be an improvement for all Motif series.

Please keep us informed if you hear from John Melas.

 

B.Minor -

However, as a MOXF user I’m also expecting something back from the Yamaha guys. Or maybe a contribution from other persons who could help clarifying why the amount of required sequencer memory for mix data in the MOXF has to be much higher than for the same data in the XF, thus lowering the possibilities for MOXF users. If this amount could be lowered by design, I’m sure that this “Sequence memory full” error would occur less often.

I hope you get a response of some sort from Yamaha, although posting here at Motifator might not be the best way to reach them.

5pinDIN:
It might be interesting to determine the size of bulk dumps of Song and Pattern Mixes from the MOXF (see page 121 of the Reference Mnaual). With the XF, either of them is 2903 bytes. That’s in contrast to the 3 kB sequencer memory used by the XF for a Song Mix, and 5 kB for a Pattern Mix. (I used MIDI-OX to capture the dumps. http://www.midiox.com/) For a Song, the memory usage is about the size of the bulk data, but it’s significantly more for a Pattern.

Thanks 5pinDIN for pointing me to that MIDI-OX tool. I installed it to verify what’s the actual size of any mix data actively dumped from the MOXF to this external tool.

To make it short, I can see the same effects with my MOXF as you also experienced with your Motif XF. Regardless if dumping a song mix or a pattern mix, if it’s related to an “empty” (unused) location or a “utilized” (e.g demo song) location, in case of the MOXF one single dump will always consist of exactly 3,013 bytes (3 kB). In every case the MIDI dump will be sent out using the same structure, having several sections with the following sizes:

13 bytes
93 bytes
51 bytes
53 bytes
49 bytes x 2
33 bytes
51 bytes
27 bytes
75 bytes x 16
85 bytes x 16
21 bytes
13 bytes

If all 128 mix dumps would be returned to the MOXF, in the best case “only” 385,664 bytes (377 kB) would be required in the MOXF sequencer memory to hold all mix data. Instead, almost twice as much memory (704 kB) is actually occupied if you’d really decide to send them all.

5pinDIN:
I don’t know what accounts for the additional Pattern Mix data memory usage, given that the bulk data is sized the same as for a Song (with the XF, anyway).

Like you I’m raising the same question why such a single mix set - apparently containing all necessary data to define a pattern mix or song mix - will actually cause more memory occupation in the built-in MOXF sequencer memory than those 3 kB.

Just to summarize the situation for the MOXF:
4.75 kB vs. 3 kB per song mix
6.25 kB vs. 3 kB per pattern mix

For me personally that doesn’t make any sense and brings me back to my initial assumption that there must be a carelessness in managing the internal MOXF sequencer memory in conjunction with any mixes. If this discrepancy in memory size is meant to be intentional, I’d really like to know why so much “additional” memory has been reserved by design. Otherwise I’d say it’s just wasted.

However, as already mentioned before, I’ll remain in contact with John Melas in order to find additional ways to optimize the situation.

B.Minor - 01 November 2014 04:21 PM

Thanks 5pinDIN for pointing me to that MIDI-OX tool. I installed it to verify what’s the actual size of any mix data actively dumped from the MOXF to this external tool.

You’re welcome. MIDI-OX can be a very helpful utility.

 

B.Minor -

To make it short, I can see the same effects with my MOXF as you also experienced with your Motif XF. Regardless if dumping a song mix or a pattern mix, if it’s related to an “empty” (unused) location or a “utilized” (e.g demo song) location, in case of the MOXF one single dump will always consist of exactly 3,013 bytes (3 kB). In every case the MIDI dump will be sent out using the same structure, having several sections with the following sizes:

13 bytes
93 bytes
51 bytes
53 bytes
49 bytes x 2
33 bytes
51 bytes
27 bytes
75 bytes x 16
85 bytes x 16
21 bytes
13 bytes

If you’d like to know what data those individual SysEx messages hold, see the MOXF Data List…
On page 120, (3-6-4)BULK DUMP. Note that the message is 13 bytes, excluding any for data.
On page 124, Bulk Dump Block table, SONG/PATTERN section. Add 13 to the byte count in the table to complete the message and match what you posted.

For example:
SONG/PATTERN-Bulk Header is shown as having a data Byte Count of 0(zero) - add 13 to get 13 total
Common is shown as Byte Count 80 - add 13 to get 93
Reverb has 38 data bytes plus 13 = 51 bytes.
Etc.

5pinDIN:
If you’d like to know what data those individual SysEx messages hold, see the MOXF Data List…
On page 120, (3-6-4)BULK DUMP. Note that the message is 13 bytes, excluding any for data.
On page 124, Bulk Dump Block table, SONG/PATTERN section. Add 13 to the byte count in the table to complete the message and match what you posted.

For example:
SONG/PATTERN-Bulk Header is shown as having a data Byte Count of 0(zero) - add 13 to get 13 total
Common is shown as Byte Count 80 - add 13 to get 93
Reverb has 38 data bytes plus 13 = 51 bytes.
Etc.

Very useful information to dig deeper into the Motif SysEx data structure. Thanks a lot for your explanations, 5pinDIN.

While yesterday I was just exploring the MIDI bulk data sent from the MOXF towards the MIDI-OX tool, today I wanted to watch what’s exactly sent from the Total Librarian application back to the MOXF, e.g. when transmitting one single song mix or pattern mix from there. Unfortunately I’m stuck now with the MIDI-OX setup.

While the MIDI-OX tool is still displaying any incoming data from the MOXF which are also correctly received by the Total Librarian tool (in menu “MIDI devices” the input port #1 - Yamaha MOXF6/MOXF8-1 is active), I’m not able to trace the outgoing MIDI path from the Total Librarian application towards the MOXF. That should also be possible by enabling the Yamaha MIDI output port #1 - Yamaha MOXF6/MOXF8-1 which I activated in the tool. However, not even after activating all Yamaha MIDI output ports #1 to #5 it will be possible to watch what’s going on - there is simply nothing displayed in the MIDI-OX window for the outgoing direction, even though the MIDI data will be received correctly by the MOXF. In the section “port mapping” inside the “MIDI devices” menu, all Yamaha ports are also mapped to the “MIDI-OX events”, so in theory this should have been enough to get it working.

5pinDIN, do you eventually have an idea what I’m missing here to set up the tool correctly in order to see the other direction? Thanks a lot.

Okay, please forget my last question. Since I’m new to this tool, I missed the fact that two monitor screens must be opened - one for the input and one for the output. Doh!

I’m very pleased to report good news today.

The identified issue with the MOXF in conjunction with received MIDI bulk dumps - namely storing back “default/empty” song/pattern mix data the same way as “allocated/used” mix data - can be avoided in the future in order not to waste any precious sequencer resources anymore.

John Melas, who was extremely committed to this topic and very helpful in every regard, has accepted my proposal which he will implement in his next update of Total Librarian. Here’s what he replied to me:

B.Minor:
> I’m not a programmer, but maybe it’s possible to have a “flag” set when
> “Receiving all...” (or just when receiving a single item) by the Total
> Librarian library whenever a pattern or song hasn’t been altered from its
> default/initialized state. Whenever such a “default” pattern or song (or
> even a whole “bank") is then selected to be sent back to the device, maybe
> the application can check the contents first in order to exclude/skip all
> “flagged” items to prevent the MOXF from allocating any “additional”
> sequencer memory for them in the MOXF.

John Melas:
You are absolutely correct! I think this would be a great update for Total Librarian! As you know I cannot even display the names of the Songs when I receive the Mix Data from the synth. Distinguish “empty” to “allocated” would be at least a big help.
Of course I can do this! This is already put in my todo list for the next update! Thank you very much for your valuable input and for your efforts to track this issue!

Best regards
John Melas

Thanks to John’s approach there will be huge savings in utilized sequencer memory, at least when exchanging mix data via SysEx messages.

However, still the general question persists why the MOXF is consuming too much sequencer memory compared to the small amount of actual data contained in such a song/pattern mix (and even compared to his big brother, the Motif XF).

Considering the pure MOXF “payload” (with the aid of Yamaha’s MIDI dump data lists and analyzing the transmitted data structure using the MIDI-OX tool), only 2,454 bytes are actually needed in order to define one single mix. Adding some necessary “overhead” bytes for MOXF internal memory managment, I think I’m not wrong claiming that 2.5 kB in total would be more than sufficient to store such a mix set completely inside the MOXF memory. Instead, currently up to 2.5 times as much memory (6.25 kB in case of pattern mixes) is consumed for each mix set.

Of course, the general problem running out of sequencer memory cannot be solved by John’s solution which will only apply to certain cases. That’s why I still hope that Yamaha is also willing to optimize their internal “housekeeping” for the sequencer data in the next F/W release, as all other user complains related to insufficient memory for MIDI track/event data would also have to be addressed.

So, I guess the ball is now back on Yamaha’s side. As there is no possibility to change the physical sequencer memory limit which is unfortunately just 896 kB by design, the only way of saving any sequencer resources would be to optimize the general memory management inside the MOXF.

5pin, I just went back and read through this thread and I saw that you responded to me back on Oct. 30.  Sorry I missed that one, I got a little lost with all the problems you and Bminor were discussing and just missed your comment to me.  Thank you for the calculations and suggestion.  I will try that and see if that helps solves the memory problem I’m having (that is… the memory problem my MOXF is having). I’m sure I’ll have my own memory problems soon enough : )

Yes, I agree Yamaha has to come up with an updated Firmware to somehow organize a better memory capability, mainly to have the possibility of increasing the sequencer memory. The current sequencer memory is simply not enough. It will be great if they can have the sequencer memory increase by using the optional Flash card memory ( I understand from all info from thus forum that this FLash card is only to store the wave samples).
I face the same problem as well, and I redo my few songs by cutting all Controllers and thinning out data ,etc ,however it improve only marginally.

I also went to another options, by playing SMF via. Usb stick. It works , but you cannot change your main live playing sounds while playing the SMF . The good thing is that , Moxf automatically resets all 16 tracks to default when a SMF stops.

Now I am thinking of using a laptop to run a SMF player ( Van Basco ) or mix down all my sequencers into Audio files (AIFF ) and play back via a laptop.

It would be great to only use Moxf alone for my performance .

Thank you to all for a great forum.

I’ve been reading through this thread and I share the frustration of encountering the “memory full” demon after recording just a handful of songs.  I agree with Jeff that memory is so cheap these days, it seems ridiculous—I would even invoke the term “design flaw”—to skimp on a feature this important to musicians and songwriters.  Personally, I would be willing to trade in several Euro Funk performances I will never use for even 1 GB more in sequencer memory.
To clarify: it is my understanding that the only viable option here is to save all songs to a back-up USB and then delete them from the MOXF, thus freeing up memory?  To recall them later for playback or editing, one must repeat the save/delete process with any exiting songs on the board, then re-load all?  This seems rather cumbersome.  Is it possible to simply save on a “per song” basis to an external location, rather than constantly saving data in bulk format?
Thanks to all for the good feedback.

B.Minor:
[...] The identified issue with the MOXF in conjunction with received MIDI bulk dumps - namely storing back “default/empty” song/pattern mix data the same way as “allocated/used” mix data - can be avoided in the future in order not to waste any precious sequencer resources anymore.

John Melas, who was extremely committed to this topic and very helpful in every regard, has accepted my proposal which he will implement in his next update of Total Librarian. [...]

I’m just returning to this topic in order to inform you about the fact that a new version of Melas Tools (V2.4.0) will be available soon.

As already mentioned few posts ago, John was so kind and developed a solution in order to keep the sequencer memory allocation as low as possible while transferring pattern or song mix data back and forth between the MOXF and the Total Librarian application.

Yesterday I had the honor to test the preliminary MOXF version and I’ve got to report that it really works as desired! Nothing more can be done from a user’s perspective to avoid unnecessary sequencer memory allocation while working with SysEx commands; the rest is still up to Yamaha in order to improve their “internal sequencer memory housekeeping”.

John Melas will release his great improvements for all Melas Tools Suites managing that kind of sequencer features, including the versions for the MOXF/MOX series as well as for the Motif XF/Motif XS series. So be prepared…