Skip to content

SONARJAVA-7283 Decouple the frontend AST from Eclipse JDT core - #6449

Closed
aurelien-coet-sonarsource wants to merge 5 commits into
masterfrom
ac/java-psi-frontend-2
Closed

aurelien-coet-sonarsource wants to merge 5 commits into
masterfrom
ac/java-psi-frontend-2

Conversation

@aurelien-coet-sonarsource

@aurelien-coet-sonarsource aurelien-coet-sonarsource commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary by Gitar

  • Frontend sessions and lifecycle:
    • Introduced FrontendSession, FrontendParser, and FrontendResources so ECJ environment cleanup is owned deterministically per session
    • Replaced JParserConfig batch and file-by-file parsing with session-based parsing and cleanup
  • Decoupled semantics from ECJ bindings:
    • Added EcjTreeBindings and EcjSemanticAttachment so trees hold lazy Supplier values instead of raw IBinding fields
    • Moved annotation and metadata conversion into EcjSymbolMetadata and JSymbolMetadata suppliers
  • Diagnostics separation:
    • Added EcjDiagnosticProvider and moved warning mapping and compiler option handling out of JProblem and JParserConfig
  • JSP compilation:
    • Added JasperCompiler to compile tag helpers with the running JDK compiler
  • Tests:
    • Added TestResourceScope and frontend lifecycle, semantic fixture, and identity tests

This will update automatically on new commits.

@hashicorp-vault-sonar-prod hashicorp-vault-sonar-prod Bot changed the title Decouple the frontend from Eclipse JDT core SONARJAVA-7283 Decouple the frontend from Eclipse JDT core Nov 2, 2026
@hashicorp-vault-sonar-prod

hashicorp-vault-sonar-prod Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

SONARJAVA-7283

@aurelien-coet-sonarsource

Copy link
Copy Markdown
Contributor Author

/pvf

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

@aurelien-coet-sonarsource ❌ Your /pvf command was not dispatched. See the failed run.

No candidate-version artifact for 7e18c5f; wait for Build to finish

@aurelien-coet-sonarsource

Copy link
Copy Markdown
Contributor Author

/pvf

@hashicorp-vault-sonar-prod

Copy link
Copy Markdown
Contributor

📊 Performance Validation complete
🆕 0 new · 🗑️ 0 lost issues
View dashboard

Projects with unexpected PVF results:

  • crate

@rombirli rombirli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to continue this review later, just adding early bugbot findings yet

for (JProblem.Type type : JProblem.Type.values()) {
if (problem.getID() == IDS.get(type)) {
LineColumnConverter.Pos start = positions.toPos(problem.getSourceStart());
LineColumnConverter.Pos end = positions.toPos(problem.getSourceStart());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

End position incorrectly uses getSourceStart(), producing zero-width warning ranges.

@gitar-bot

gitar-bot Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Code Review ✅ Approved 2 closed / 2 findings

🟡 Medium risk · Refactoring Java semantic bindings into lazy suppliers changes type and symbol resolution across the frontend, so consumers could receive incorrect analysis results.

Decouples the frontend from Eclipse JDT core by replacing shared-tree compiler bindings with lazy semantic suppliers, adding session-based parser lifecycle and a separate diagnostics provider. Resolved: JSP tag-file compilation no longer fails on JRE-only scanners. Open minor suggestion: undefined-type reporting (CompilationUnitTreeImpl.undefinedTypes) will silently go empty if combined with #6437's SemSitter path, so the frontend should populate it and #6437's translator should copy its undefined types into it.

✅ 2 closed
✅ Bug: JSP tag-file compilation now needs a JDK; fails on JRE-only scanners

📄 java-jsp/src/main/java/org/sonar/java/jsp/JasperCompiler.java:43-46 📄 java-jsp/src/main/java/org/sonar/java/jsp/JasperOptions.java:158 📄 sonar-java-plugin/pom.xml:247-249
JasperOptions.getCompilerClassName() now returns JasperCompiler, and sonar-java-plugin/pom.xml drops Jasper's JDTCompiler* classes from the shaded jar. JasperCompiler.generateClass uses ToolProvider.getSystemJavaCompiler(). On a runtime without the jdk.compiler module, such as a plain JRE (the kind scanners usually run on or get provisioned with), that call returns null, and the method throws JasperException("A JDK compiler is required to process JSP tag files").

Jasper compiles tag files to classes so it can introspect them. Before this change the bundled ECJ did that work, so it ran on any JRE. Now every JSP that uses a tag file fails inside Jasper.transpile's per-file catch and only logs "Some JSP pages failed to transpile". Those JSPs then get no analysis on JRE environments.

Fix options: keep an in-process fallback (ECJ/JDTCompiler) for when no system compiler exists, or degrade without failing the page. At minimum, document the new JDK requirement.

Closed: Cross-PR: Undefined-type reporting lost or broken when combined with #6437

📄 java-frontend/src/main/java/org/sonar/java/ast/JavaAstScanner.java:172 📄 java-frontend/src/main/java/org/sonar/java/model/JavaTree.java:166-168 📄 java-frontend/src/main/java/org/sonar/java/model/JavaTree.java:202-204 📄 java-frontend/src/main/java/org/sonar/java/model/JParser.java:356
Breaks when #6437's SemSitter translation path is merged onto a base that has this PR, and a file is then scanned through JavaAstScanner.simpleScan.

How

  1. This PR moves undefined types out of JSema and onto the tree: it adds CompilationUnitTreeImpl.undefinedTypes and its accessor (java-frontend/src/main/java/org/sonar/java/model/JavaTree.java:168,202-204), removes JSema.undefinedTypes(), and changes the sema field's type to the Sema interface (JavaTree.java:166).
  2. JavaAstScanner.simpleScan now reads ast.undefinedTypes() (java-frontend/src/main/java/org/sonar/java/ast/JavaAstScanner.java:172). The only code that fills that set is JParser.convert (JParser.java:356).
  3. SONARJAVA-7236 Score AST and semantic mismatches against ECJ #6437's SemSitter translator only sets unit.sema = sema, keeps the set inside its own JSema, and its JavaAstScanner.java:167 still calls ast.sema.undefinedTypes().
  4. If the merge takes this side, the SemSitter path always reports an empty set, so collectUndefinedTypes and the typeErrorCount telemetry silently stop. If it takes the other side, the code doesn't compile, because Sema has no undefinedTypes().

Fix

  • This PR: Document that the frontend is now responsible for filling CompilationUnitTreeImpl.undefinedTypes. Optionally, add a non-ECJ seam (for example, a package-private method that adds undefined types) that another frontend can call.
  • SONARJAVA-7236 Score AST and semantic mismatches against ECJ #6437: In the SemSitter translator, copy the undefined types collected by its JSema into unit.undefinedTypes. Switch JavaAstScanner and the tests to ast.undefinedTypes().

Compared dc20e3c with #6437 at 1df8a88

Verification Rules 1/1 met PRs 2 inspected

Functional validation No results

Rules 1 passed 2 not applicable

✅ Repository license compliance

Broader impact PRs 2 inspected
Cross-PR 2 PRs inspected 0 open impacts

PR 6416: SONARJAVA-7252 Decouple the Java frontend from Eclipse JDT core Inspected
PR 6432: SONARJAVA-7259 Introduce a compiler-neutral frontend backend with scoped resource management Inspected

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqube-next

sonarqube-next Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

@aurelien-coet-sonarsource aurelien-coet-sonarsource changed the title SONARJAVA-7283 Decouple the frontend from Eclipse JDT core SONARJAVA-7283 Decouple the frontend AST from Eclipse JDT core Nov 3, 2026

@rombirli rombirli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't review yet from JParser to the end, will do it later

private ListTree<TypeTree> permittedTypes;
@Nullable
public ITypeBinding typeBinding;
public Supplier<Symbol.TypeSymbol> typeSymbol = SemanticValues.UNKNOWN_TYPE_SYMBOL;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this public non-final supplier field looks like a code smell to me, a property with getter/setter would be clearer

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I don't know why these fields were public in the first place, but let's move them behind getter/setters.

import javax.annotation.Nullable;
import org.eclipse.jdt.core.dom.IBinding;

final class EcjSemanticAttachment<B extends IBinding, T> implements Supplier<T> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If i'm correct we can constrain T to Symbol and its subclasses only

Suggested change
final class EcjSemanticAttachment<B extends IBinding, T> implements Supplier<T> {
final class EcjSemanticAttachment<B extends IBinding, T extends Symbol> implements Supplier<T> {

private SyntaxToken endToken;

public IVariableBinding variableBinding;
public Supplier<Symbol> variableSymbol = SemanticValues.UNKNOWN_SYMBOL;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again public mutable fields seem not very conventional to me

private List<AnnotationTree> annotations;

public IBinding binding;
public Supplier<Symbol> symbolSupplier = SemanticValues.UNKNOWN_SYMBOL;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.


@Nullable
public IMethodBinding methodBinding;
public Supplier<Symbol.MethodSymbol> methodSymbol = SemanticValues.UNKNOWN_METHOD;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As i see it everywhere, could JavaTree become a generic class JavaTree, and contain a symbol property (getter+setter)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the issue is that not all classes that inherit from JavaTree have an associated symbol, so this would add a field and getter/setter with some default value to a lot of classes where it doesn't actually make sense.

import org.sonar.java.model.expression.MethodInvocationTreeImpl;
import org.sonar.java.model.pattern.AbstractPatternTree;

final class EcjTreeBindings {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for this one

public JSema sema;
public Sema sema;

final Set<JProblem> undefinedTypes = new HashSet<>();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is default visibility expected here?
Should it be public instead, packages might move

Comment on lines +202 to +205
public Set<JProblem> undefinedTypes() {
return Collections.unmodifiableSet(undefinedTypes);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would go for either

  • property style :
    • getter return immutable view on the set as there
    • public functions to edit the set (add, empty, addall...)
  • public field style
    • just a public final field

But this undefinedTypes currently mixes public immutable getter, with default-visible final field

Comment on lines +91 to +111
private Object convertAnnotationValue(Object value) {
return EcjSymbolMetadata.convertAnnotationValue(sema, value);
}
}

static Object convertAnnotationValue(JSema sema, Object value) {
return switch (value) {
case IVariableBinding iVariableBinding -> sema.variableSymbol(iVariableBinding);
case ITypeBinding iTypeBinding -> sema.typeSymbol(iTypeBinding);
case IAnnotationBinding iAnnotationBinding -> sema.annotation(iAnnotationBinding);
case Object[] a -> {
Object[] result = new Object[a.length];
for (int i = 0; i < a.length; i++) {
result[i] = convertAnnotationValue(sema, a[i]);
}
yield result;
}
case null, default -> value;
};
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 return types are too broad to me, the return type could be symbol to be bounded and easier to understand

Comment on lines +73 to +77
@Override
@Nullable
public Object defaultValue() {
return EcjSymbolMetadata.convertAnnotationValue(sema, methodBinding().getDefaultValue());
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the return type could also be bounded to Symbol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants