Improvements to FreeJ2ME's game compatibility by adding some missing classes (redone) by AShiningRay · Pull Request #145 · hex007/freej2me

AShiningRay · 2022-06-11T19:35:11Z

Copying the PR message from the previous similarly-named one: More general improvements to the game compatibility, this time by adding some classes not available to them prior and working around a few weird behaviors that some games relied on, enabling many to go beyond the boot sequence and even reach near-perfect gameplay now. ~~The screen pointer on the libretro frontend was also fixed, since it wasn't pointing at the correct coordinates when the screen was rotated, which many Touchscreen games require.~~ - Moving this one into a separate commit, just like the "negative index" fix.

But this time, those new implementations and stubs are clean and based off of Java's ME JSR Community Process docs, since many can't be found on the main javadocs pages.

The docs: https://jcp.org/en/jsr/platform?listBy=1&listByType=platform

Edit: This one fixes #148, although the issue was created after this PR.

Games like 3D Pinball Timeshock use the -1 index as a record ID for some reason, so in cases where the index is lower than 0, the code truncates the index to 0. Allowing Timeshock to boot, but failing a bit later due to the current 3D implementation.

Before, when the screen was rotated on the libretro frontend, the pointer coordinates in the java file were reversed correctly, but some extra calculations to account for the 90 degrees counterclockwise rotation were not done, making the pointer be "reversed" in the y axis, so the lower the pointer was in the screen, the higher the y value it had in the rotated mode. Now it works as intended.

Some games rely on those classes to be present, like my 320x240 copy of Asphalt Nitro. The 240x320 version of the same game doesn't seem to use them however. All files are based off of Java's ME JSR Community Process docs.

Those files allow games like 3D Formula Extreeme! to boot. Code based off of Java's ME JSR Community Process docs as well.

recompileorg

I haven't looked at the javax.microedition.io.file classes yet, so I'd double-check those against the docs you have.

It doesn't seem like that package has an unusual history. Here's what I found for it with a quick search. It may be worth comparing against what you have:

https://blackberry.com/developers/docs/5.0.0api/javax/microedition/io/file/package-summary.html

recompileorg · 2022-06-11T21:00:20Z

+*/ 
+package javax.bluetooth;
+
+public abstract class DeviceClass


Why is this declared abstract?

https://docs.oracle.com/javame/config/cldc/opt-pkgs/api/bluetooth/jsr082/javax/bluetooth/DeviceClass.html

recompileorg · 2022-06-11T21:02:44Z

+*/ 
+package javax.bluetooth;
+
+public abstract class DiscoveryAgent


Why is this declared abstract?

javax.microedition.lcdui.Canvas is an example of a class that needs to be abstract. You'll see that the javadocs clearly label it as such:
https://docs.oracle.com/javame/config/cldc/ref-impl/midp2.0/jsr118/javax/microedition/lcdui/Canvas.html

recompileorg · 2022-06-11T21:03:36Z

+*/ 
+package javax.bluetooth;
+
+public abstract interface DiscoveryListener


Why is this declared abstract?

https://docs.oracle.com/javame/config/cldc/opt-pkgs/api/bluetooth/jsr082/javax/bluetooth/DiscoveryListener.html

recompileorg · 2022-06-11T21:05:53Z

+import javax.microedition.io.Connection;
+
+
+public abstract class LocalDevice 


Why is this declared abstract?

https://docs.oracle.com/javame/config/cldc/opt-pkgs/api/bluetooth/jsr082/javax/bluetooth/LocalDevice.html

recompileorg · 2022-06-11T21:06:36Z

+
+    public abstract String toString();
+}
+


While it doesn't really matter in Java, there's no reason not to add a newline at the end.

recompileorg · 2022-06-11T21:07:01Z

+*/
+package javax.bluetooth;
+
+public abstract class UUID 


Why is this declared abstract?

https://docs.oracle.com/javame/config/cldc/opt-pkgs/api/bluetooth/jsr082/javax/bluetooth/UUID.html

recompileorg · 2022-06-11T21:08:10Z

+    public void truncate(long byteOffset);
+
+    public long usedSize();
+}


While it doesn't really matter in Java, there's no reason not to add a newline at the end.

recompileorg · 2022-06-11T21:12:49Z

 									else
 									{
-										Mobile.getPlatform().pointerDragged(mousey, mousex);
+										Mobile.getPlatform().pointerDragged(-mousey+lcdWidth, mousex);


This isn't important, but lcdWidth-mousey makes the intent more obvious.

Oh that was because i wrote the properties of doing a 90 deg counterclockwise rotation, then corrected the pointer position it by adding the width, perhaps the translation to code was a bit too direct. But yes, that recommendation makes it much easier to read. I'll be updating it later.

recompileorg · 2022-06-11T21:17:54Z

I've based my review on JSR 82:
https://docs.oracle.com/javame/config/cldc/opt-pkgs/api/bluetooth/jsr082/javax/bluetooth/package-summary.html

AShiningRay · 2022-06-12T02:34:52Z

Oh shoot, i forgot to mark this one as a draft. I'm really Sorry!

The reason they are abstract for now is because i just stubbed them as much as i could to test if they were already causing any effect. i¨ll be converting it to a draft shortly.

The implementations were mostly in line with the javadocs as of the previous commit. This one makes them nearly identical to the docs.

recompileorg · 2022-06-12T05:00:17Z

+    DataInputStream openDataInputStream() throws SecurityException, IllegalModeException;

-    public DataOutputStream openDataOutputStream();
+    @Override /* Throws IOException in the docs, but not on the overriden class */


" Throws IOException in the docs, but not on the overriden class" What does this mean? What other class? Remember that this is an API. That means it is entirely separate from any specific application that uses it.

It means that this DataOutputStream method is described as throwing an IOException in the javadocs. However, our current implementation (the compiler at least complained about not overriding a function with the same name as that) doesn't. I'll clean it up later when i do a second pass on all those files.

recompileorg · 2022-06-12T05:02:40Z

 import java.util.Enumeration;

-public abstract class FileSystemRegistry 
+public class FileSystemRegistry extends Object


extends Object is unnecessary as all Java classes extend Object.

Yep, that was just to make an exact transcription of the docs, i still need to clean some things up.

This folder is a prerequisite for some of the incoming implementations of the recent bluetooth files, which even in their mostly stubbed state still needs these obex files to be present.

The code in them should now be mostly identical to the javadocs.

AShiningRay · 2022-06-13T22:51:28Z

Hmm... after looking at all the files again, seems this PR is ready for review... at least to me.

recompileorg

I didn't really give it a complete once-over. There's enough to keep you busy anyway. Also we try to use tabs for indenting, spaces for alignment. Your editor should be able to do that conversion automatically.

recompileorg · 2022-06-13T23:29:58Z

+
+    public boolean equals(Object value) { return false; }
+
+    public int hashCode() { return uuid.hashCode(); }


This line has a lot of problems. First, uuid is null and will never be set because, even though it's public by default, no game exists that would try to set this property as it's not part of the standard. Even if it was initialized, calling this method will cause a stack overflow. Java does not have tail call optimization, but even if it did this would still be an infinite loop.

If you're just stubbing this out:
The rules for this, believe it or not, will let you get away with returning a the same integer for every value.
From: https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#hashCode()
"It is not required that if two objects are unequal according to the equals(java.lang.Object) method, then calling the hashCode method on each of the two objects must produce distinct integer results. However, the programmer should be aware that producing distinct integer results for unequal objects may improve the performance of hash tables."

However "If two objects are equal according to the equals(Object) method, then calling the hashCode method on each of the two objects must produce the same integer result. " This is probably why this overrides Object.hashCode

recompileorg · 2022-06-13T23:33:46Z

+public class UUID
+{
+
+	UUID uuid;


This should be declared private and it shouldn't be a UUID -- that's the thing you're defining here, after all. The way you try to use this later also causes a few problems.

I'd recommend using 4 longs, using each as a 32-bit unsigned integer to store the UUID. It'll make things a lot simpler.

recompileorg · 2022-06-13T23:34:58Z

+
+    public int hashCode() { return uuid.hashCode(); }
+
+    public String toString() { return BASE_UUID_VALUE; }


This should return whatever it is you're using to store the uuid converted to a string (hexadecimal) this could be a 16, 32 or 128 bit value, but this should always return the 128 bit version. An odd quirk, leading zeros should not be included, meaning a 128-bit uuid will not necessarily turn in to a 32 character long string.

I know you're just stubbing this out, but it's not a very complicated class to just implement.

recompileorg · 2022-06-13T23:47:09Z

+    public UUID(String uuidValue, boolean shortUUID) throws IllegalArgumentException, NullPointerException,
+	NumberFormatException { System.out.println("UUID Value:" + uuidValue); }
+
+    public boolean equals(Object value) { return false; }


You can just check strings returned from toString using the String.equals method. Be aware that you'll need to cast value to a UUID first.

recompileorg · 2022-06-14T01:55:57Z

+{
+
+	UUID uuid;
+	private static final String BASE_UUID_VALUE = "0x0000000000001000800000805F9B34FB";


This is very wrong. First, the "0x" prefix shouldn't be included in a string representing a UUID. Second, storing this as a string isn't terribly useful. It would be better spread this out over 4 longs. Remember that the base is used to "promote" 16 and 32-bit UUIDs to 128 bits. (The toString and equals methods require this.)

Very well, i'll be applying those changes, including the others above.

recompileorg · 2022-06-14T02:42:05Z

+public class IllegalModeException extends RuntimeException 
+{
+
+	public IllegalModeException() { } /* Constructs an instance of the class with its stack trace */


I'm not sure this comment or its twin are helpful.

Those were just copied from the docs, but they're a bit vague, yes. Makes more sense to drop them altogether.

recompileorg · 2022-06-14T02:47:33Z

 			//System.out.println("> Next Record ID (idx:"+index+" cnt:"+count+")");
 			if(keepupdated) { rebuild(); }
 			if(index>=count) { throw(new InvalidRecordIDException()); }
+			if(index < 0) return elements[0]; // If an invalid index is received, truncate it to 0.


That's not what 'truncate' means. Also, how would index ever become less than 0?

Yeah, i agree that using "truncate" there doesn't make sense.

And as for the index becoming less than 0, a game i tested some days ago (Pinball 3D Timeshock) does that:

It's the game that i fixed by adding that index check. I have no idea why it tries to access a recordId on index -1, but forcing it to move to index 0 allowed it to save its data afterwards:

Previously it would only save the "highScore" data, not the "CameraType".

Still, that might not be the best solution to cases like those, i'm open to suggestions.

That's interesting. I'd like to figure out what happened. There shouldn't be any way to get index below zero. We should look for any places where index is reduced and put the check there (or force it to zero).

Decided to un-comment every single System.out line in the RecordStore class to see what was going on while it tried to save and load the rms files, and it turns out that the game doesn't even decrement the index while reading, it just goes for a negative one an an invalid ID right out of the gate, at least it's what the logs tell me (either that, or i'm interpreting the messages incorrectly):

It looks like there are a few places in enumeration where it could potentially go below zero. previousRecord and rebuild both assume count is greater than zero. I'm surprised this escaped notice until now. Count should also be initialized in the constructor... ugh... I'll see about updating this later today.

I can do those if you want, should help keeping things cleaner as they'll all be in the same PR.

Edit: Hmm... i think i fixed it. Just initialized count as 0 in the enumeration constructor and added a check for when index < 0 in the rebuild() method. I commented that check i placed inside nextRecordId() and so far it still works fine in Pinball 3D Timeshock:

I decided to test DOOM II RPG as well since it uses rms a bit more extensively compared to other games like asphalt 6 (still works fine) and so far it's still ok:

Sure, those other games shouldn't be affected by those changes, but it's always good to do a little testing.

Don't worry about the number of pull requests. I'd rather see a bunch of smaller ones than a big omnibus. If something needs reverted, it's helpful if the PRs are more focused. Ideally, a PR will address a specific issue. It's not uncommon for more experienced contributors to create an issue right before they submit a PR that addresses it.

It also makes the review process simpler. It takes a lot of time to go over these, and I know I'm not spending nearly enough time on most of the files just because of how many there are. This one alone has something like 22 files changed. I'd bet that most of these files would have already been approved had they been standalone or in smaller groups.

Remember that most of this project is tending toward an ideal. Just like an emulator has the hardware as a standard, this project has the standards as a standard. The goal isn't to get a game to work, but to get closer to that ideal. What matters is correctness, above all. It wasn't uncommon early on to have correct changes break games.

Don't try to make the game work. That's the easy thing to do, but it's usually wrong. It's better to find out what problems with our implementation are keeping the game from working.

Yeah i'm not worried about making things work with that change above. I just tried to do a few patches to the RecordStore file based on your pointers and it seems to have worked correctly.

And i'll definitely keep that smaller PR advice in mind.

recompileorg · 2022-06-14T02:48:39Z

+{
+
+    PasswordAuthentication onAuthenticationChallenge(String description, 
+    boolean isUserIdRequired, boolean isFullAccess);


Why split this across two lines?

I was trying to make the lines shorter, kinda like it's done in older C standards. Maybe it isn't useful here, so i'll get those back to one longer line.

The UUID implementation might still be incorrect, i'm basing it almost entirely on the data from the docs.

AShiningRay · 2022-06-14T23:11:49Z

Just gave it a shot on implementing the UUID (alongside many adjustments to the other files added in this PR). If there's anything else to be done, or the implementation is incorrect, please share. I'm not used to doing those manipulations used in the UUID.

Instead of checking and working around cases where the rms index would be lower than zero, it's better to simply find a way to make sure it won't ever go below zero at all. This reverts commit 3ee6a3c.

recompileorg · 2022-06-28T03:45:40Z

+	@Override
+	public String toString()
+	{
+		String uuidString = Long.toHexString(UUID1) + Long.toHexString(UUID2) + Long.toHexString(UUID3) + Long.toHexString(UUID4);


This won't do what you want.

Consider the following values for UUID1-4: 0x00000000, 0x0011223344, 0x00667788, 0x0000BBCC

The expected output from toString is: "11223344006677880000BBCC"

Your function will produce: "11223344667788BBCC" as Long.toHexString will not prepend leading zeros.

Bah, it figures I wouldn't catch a mistake until I posted this. Ignore the extra 8 bits at the front of UUID2.

Makes sense. I think i can just format those long values to a 8 char hex string through String.format, not needing to over-complicate stuff by getting that toHexString result and manipulating it.

recompileorg · 2022-06-28T03:45:56Z

+	public String toString()
+	{
+		String uuidString = Long.toHexString(UUID1) + Long.toHexString(UUID2) + Long.toHexString(UUID3) + Long.toHexString(UUID4);
+		return uuidString.toUpperCase().replaceFirst("^0+(?!$)", "");


According to the bluecove docs (which seem more complete than the JSR 82 docs from Oracle) Only leading zeros should be removed.

I don't think leaving that more robust regex in would cause problems given the expected UUID format, but i'll change it to remove only the zeros nonetheless.

This is a little more complicated than I expected. I wrote this up last night, along with a few tests and it seems to be working correctly. It's pretty heavily commented, which should be helpful as while the conversions to and from strings are very simple, they can be a bit tricky if you're not used to bit twiddling.

I'll give myself a B- on the quality scale. There's room for improvement here, but 'where' is not super obvious to me at the moment and I really should be doing other things right now.

UUID.zip

Mine currently works as well after i dropped those toHexString methods and adjusted the value space of the baseUUID variables. But things such as making the UUID an array are good ideas, i'll be incorporating some of those changes into my code.

@recompileorg

@recompileorg had some suggestions in the form of code, and a few of the implementations could be ported over to this current one.

No idea how that empty first line escaped my sight for so long... Should be formatted correctly now.

…ble." This reverts commit 0522ab4.

This reverts commit 35cb096.

AShiningRay · 2022-06-29T21:52:52Z

Cleaned this PR up a bit. Now it only has those new files. The pointer, rms negative index and a few other minor fixes were moved into their own pull requests.

recompileorg · 2022-06-30T01:35:15Z

+
+		if (shortUUID)
+		{
+			String formattedUuidValue = String.format("%08X", uuidValue);


It's crashing on UUID t2 = new UUID("1234", true);

>java driver test... expected: 4660 hashcode: 4660 expected: 123400001000800000805F9B34FB toString: 123400001000800000805F9B34FB Exception in thread "main" java.util.IllegalFormatConversionException: x != java.lang.String at java.base/java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4426) at java.base/java.util.Formatter$FormatSpecifier.printInteger(Formatter.java:2938) at java.base/java.util.Formatter$FormatSpecifier.print(Formatter.java:2892) at java.base/java.util.Formatter.format(Formatter.java:2673) at java.base/java.util.Formatter.format(Formatter.java:2609) at java.base/java.lang.String.format(String.java:3292) at UUID.<init>(UUID.java:49) at test.<init>(test.java:13) at driver.<init>(driver.java:10) at driver.main(driver.java:5)

Hmm... i was confusing that uuidValue with the one in the constructor above, treating it as a long when i should be treating as a string instead. Changes incoming.

recompileorg · 2022-06-30T02:21:23Z

+	{ 	
+		int length = uuidValue.length();
+
+		if(uuidValue == null || length==0 || length>32 || (shortUUID && length>8) )


linLe 41 should throw a NullPopinterException if uuidValue is null. There isn't a reason to check here.

Agreed, it's a bit redundant after eyeing the code as a whole.

AShiningRay added 4 commits June 11, 2022 16:33

javax: io: Implement the javax.microedition.io.file.* classes.

255da6a

Some games rely on those classes to be present, like my 320x240 copy of Asphalt Nitro. The 240x320 version of the same game doesn't seem to use them however. All files are based off of Java's ME JSR Community Process docs.

javax: bluetooth: Stub some missing classes and interfaces.

f34c901

Those files allow games like 3D Formula Extreeme! to boot. Code based off of Java's ME JSR Community Process docs as well.

recompileorg requested changes Jun 11, 2022

View reviewed changes

AShiningRay marked this pull request as draft June 12, 2022 02:35

AShiningRay added 2 commits June 12, 2022 01:13

javax: io: Update the file/ folder.

ac0f2dc

The implementations were mostly in line with the javadocs as of the previous commit. This one makes them nearly identical to the docs.

freej2me: libretro: Make the pointer position calc more readable.

0522ab4

recompileorg reviewed Jun 12, 2022

View reviewed changes

AShiningRay added 5 commits June 12, 2022 03:48

src: javax: Stub the obex folder.

2086bc4

This folder is a prerequisite for some of the incoming implementations of the recent bluetooth files, which even in their mostly stubbed state still needs these obex files to be present.

javax: bluetooth: Finish the file additions.

f9f113f

The code in them should now be mostly identical to the javadocs.

javax: bluetooth: Some cleanups to the files.

dc1e467

javax: io: Cleanups to the file/ folder.

424df51

src: javax: Minor cleanup on the obex folder.

d242d61

AShiningRay marked this pull request as ready for review June 13, 2022 22:51

AShiningRay requested a review from recompileorg June 14, 2022 02:38

recompileorg requested changes Jun 14, 2022

View reviewed changes

AShiningRay added 3 commits June 14, 2022 19:03

javax: io: More cleanups and adjustments to the file/ folder.

30ea5f0

javax: obex: More adjustments to the obex files.

c14d5c4

javax: bluetooth: Clean up LocalDevice and implement UUID

2f6ca2c

The UUID implementation might still be incorrect, i'm basing it almost entirely on the data from the docs.

Revert "Truncate record IDs to 0 in some cases."

4e0cab9

Instead of checking and working around cases where the rms index would be lower than zero, it's better to simply find a way to make sure it won't ever go below zero at all. This reverts commit 3ee6a3c.

AShiningRay requested a review from recompileorg June 25, 2022 23:54

recompileorg requested changes Jun 28, 2022

View reviewed changes

javax: bluetooth: UUID: Fix some uuid conversion problems.

08f6a91

AShiningRay requested a review from recompileorg June 28, 2022 16:04

AShiningRay added 3 commits June 29, 2022 11:30

javax: bluetooth: UUID: Improve the code with a few ideas.

c40e573

@recompileorg had some suggestions in the form of code, and a few of the implementations could be ported over to this current one.

javax: bluetooth: UUID: Fixing a small typo.

bcd09cc

javax: bluetooth: UUID: Clearing an empty line at the start.

25da782

No idea how that empty first line escaped my sight for so long... Should be formatted correctly now.

AShiningRay mentioned this pull request Jun 29, 2022

Certain io.file.* and bluetooth.* classes and interfaces are absent #148

Open

AShiningRay changed the title ~~Improvements to FreeJ2ME's game compatibility, and a screen pointer fix when rotated on libretro (redone)~~ Improvements to FreeJ2ME's game compatibility (redone) Jun 29, 2022

AShiningRay changed the title ~~Improvements to FreeJ2ME's game compatibility (redone)~~ Improvements to FreeJ2ME's game compatibility by adding some missing classes (redone) Jun 29, 2022

AShiningRay added 2 commits June 29, 2022 17:51

Revert "freej2me: libretro: Make the pointer position calc more reada…

805b073

…ble." This reverts commit 0522ab4.

Revert "freej2me-lr: Fix rotated pointer coordinates."

5dda662

This reverts commit 35cb096.

recompileorg reviewed Jun 30, 2022

View reviewed changes

javax: bluetooth: UUID: Fixing one of the UUID constructors.

83fe551

AShiningRay requested a review from recompileorg June 30, 2022 13:41

javax: bluetooth: UUID: Adjusting the UUID variables' types.

929af84

AShiningRay mentioned this pull request Jul 6, 2022

Stubbing the rest of JSR 075 (PIM) #154

Merged

		import javax.microedition.io.Connection;


		public abstract class LocalDevice


		public boolean equals(Object value) { return false; }

		public int hashCode() { return uuid.hashCode(); }


		public int hashCode() { return uuid.hashCode(); }

		public String toString() { return BASE_UUID_VALUE; }

Conversation

AShiningRay commented Jun 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

recompileorg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

recompileorg commented Jun 11, 2022

Uh oh!

AShiningRay commented Jun 12, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AShiningRay Jun 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AShiningRay commented Jun 13, 2022

Uh oh!

recompileorg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AShiningRay Jun 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AShiningRay Jun 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AShiningRay commented Jun 11, 2022 •

edited

Loading

AShiningRay Jun 12, 2022 •

edited

Loading

AShiningRay Jun 14, 2022 •

edited

Loading

AShiningRay Jun 14, 2022 •

edited

Loading