Prismacloud_aws_detection_enhancement

[Rajamani R (raj-axe)]

• various additional detections enhancement in the support queries
• changes to limits in config query to pull more data

[Ian Hellen (ianhelle)]

Hmm. it occurred to me that the yaml format you are using might cause some issues. There seem to be 2 tests that are failing in Py3.9 and Py3.11 pipelines.
I'm kind of OK to get rid of or loosen the json_schema check.
You should check the tests/data/queries/test_query_files.py failure though.
If it is not trivial to fix, I suggest that we just bypass the prismacloud queries in the package tests.

In tests/common/test_pkg_config.py, something like adding an exception for prismacloud in the for loop:

    if "prismacloud" in str(query_file):
        continue

like this

def test_load_default():
    """Test load default settings."""
    settings = pkg_config._settings
    check.is_in("QueryDefinitions", settings)
    check.is_in("Default", settings["QueryDefinitions"])
    check.equal(1, len(settings["QueryDefinitions"]["Default"]))
    for path in settings["QueryDefinitions"]["Default"]:
        check.is_true(type(path), str)
        path = (
            Path(pkg_config.__file__).resolve().parent.parent.joinpath(f"data/{path}")
        )
        check.is_true(path.is_dir())
        for query_file in path.rglob("*.yaml"):
            if "prismacloud" in str(query_file):
                continue
            validate_queries_file_structure(query_file)

[Rajamani R (raj-axe)]

Hmm. it occurred to me that the yaml format you are using might cause some issues. There seem to be 2 tests that are failing in Py3.9 and Py3.11 pipelines. I'm kind of OK to get rid of or loosen the json_schema check. You should check the tests/data/queries/test_query_files.py failure though. If it is not trivial to fix, I suggest that we just bypass the prismacloud queries in the package tests.

In tests/common/test_pkg_config.py, something like adding an exception for prismacloud in the for loop:

    if "prismacloud" in str(query_file):
        continue

like this

def test_load_default():
    """Test load default settings."""
    settings = pkg_config._settings
    check.is_in("QueryDefinitions", settings)
    check.is_in("Default", settings["QueryDefinitions"])
    check.equal(1, len(settings["QueryDefinitions"]["Default"]))
    for path in settings["QueryDefinitions"]["Default"]:
        check.is_true(type(path), str)
        path = (
            Path(pkg_config.__file__).resolve().parent.parent.joinpath(f"data/{path}")
        )
        check.is_true(path.is_dir())
        for query_file in path.rglob("*.yaml"):
            if "prismacloud" in str(query_file):
                continue
            validate_queries_file_structure(query_file)

apologies for the long silence Ian, I finally had a block of time to address this.

What I’ve done

• Prisma Cloud YAMLs fixed
  Added explicit metadata: {} blocks (or real metadata) and removed empty literal-blocks that were resolving to null.
• Tests pass without skips locally

Let me know if you’d like any additional tweaks. thank you

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[Rajamani R (raj-axe)]

Hmm. it occurred to me that the yaml format you are using might cause some issues. There seem to be 2 tests that are failing in Py3.9 and Py3.11 pipelines. I'm kind of OK to get rid of or loosen the json_schema check. You should check the tests/data/queries/test_query_files.py failure though. If it is not trivial to fix, I suggest that we just bypass the prismacloud queries in the package tests.

In tests/common/test_pkg_config.py, something like adding an exception for prismacloud in the for loop:

    if "prismacloud" in str(query_file):
        continue

like this

def test_load_default():
    """Test load default settings."""
    settings = pkg_config._settings
    check.is_in("QueryDefinitions", settings)
    check.is_in("Default", settings["QueryDefinitions"])
    check.equal(1, len(settings["QueryDefinitions"]["Default"]))
    for path in settings["QueryDefinitions"]["Default"]:
        check.is_true(type(path), str)
        path = (
            Path(pkg_config.__file__).resolve().parent.parent.joinpath(f"data/{path}")
        )
        check.is_true(path.is_dir())
        for query_file in path.rglob("*.yaml"):
            if "prismacloud" in str(query_file):
                continue
            validate_queries_file_structure(query_file)

The initial mypy failures occurred because this branch was not up to date with main, so it still reflected pre-existing type errors in unrelated modules (auth/keyvault_client.py and context/azure/azure_data.py). These were not introduced by the Prisma Cloud changes in this PR. The branch has now been rebuilt on the latest main to ensure the diff only contains the intended changes.

can we rerun the pipeline