When I opened the very heavy enclosure, I found two boards placed on top of each other. The upper layer of boards appears to be the VRAM area directly connected to the graphics CRT port. The lower layer board contains the MC68000; if you look around the CHARACTER CRT port, you can see the letters on the board that correspond to the 25 pins of the CHARACTER CRT port.
1pin -> GND
2pin -> R(ed)
3pin -> G(reen)
4pin -> B(lue)
5pin -> BW (Black/White)
Further examination with an oscilloscope revealed the following.
6pin -> 56Hz Clock singnal
7pin -> 29.8kHz Clock signal
I guessed that these two signals were V-Sync and H-Sync. Now that I had this information, I could output the video to the monitor. I immediately tried to make a simple cable using the female VGA connector (Dsub15pin). However, it did not work.
Disappointed, I opened the enclosure again to observe the board. What I found was the HD46505SP, which was commonly used as a CRTC at that time.
There should be V-Sync and H-Sync signals coming out from pins 40 and 39 of this IC, respectively. When I checked with an oscilloscope, I found that the signals were true of the same period as pins 6 and 7 of the DB25. At this point, I looked at both signals and found that the signal on HD46505SP was positive logic and the signal on pin 25 was negative logic.
As far as I remember, the H-sync and V-sync of the VGA signal should have been positive logic. As a test, I connected the HD465055SP to the monitor with pins 2, 3, and 4 of the DB25 pin as RGB of the VGA signal and each pin of the HD465055SP as V-Sync and H-Sync, and it was perfect! The image was output!
To summarize, the video signals of the CHARACTER CRT port of the M68MX are configured as follows
Pin 2 -> R
Pin 3 -> G
Pin 4 -> B
Pin 5 -> B/W
Pin 6 -> V-sync (negative logic) 56Hz
Pin 7 -> H-sync (negative logic) 29.8kHz
If you are using only the CHARACTER CRT port to create VGA signals, you will need to insert inverters on pins 6 and 7.