[Core feature] Add support Literal Transformer

[BarryWu0812]

Tracking issue

Closes flyteorg/flyte#6503

Why are the changes needed?

Support to serialize Literal to a literal primitive

What changes were proposed in this pull request?

Add a Literal transformer

How was this patch tested?

Unit test
and
Run the below workflow

from flytekit import task, workflow, ImageSpec
import flytekit
from typing import Literal

@task
def my_task(feature: Literal["outcome", "income"]) -> str:
    return f"feature: {feature}"

@workflow
def wf(feature: Literal["outcome", "income"]) -> str:
    return my_task(feature=feature)

Screenshots?

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Summary by Bito

This pull request introduces a new Literal transformer in the Flyte framework, enhancing the type engine's ability to serialize Literal types into primitive types. It improves data type handling and usability in type management, and includes methods for converting between Literal and Python values. Unit tests have been added to verify the new functionality.

[welcome]

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

• Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
• Sign off your commits (Reference: DCO Guide).

[kumare3]

What is the goal of a literal type? Is it to provide an enum like behavior or just a weird way of passing primitives

[BarryWu0812]

#3304 (comment)

Currently, the Literal will be serialized to a pickle file instead of proto, and it would be nice to have the typing.Literal parameter also render as string in the UI.

Before, the input in UI is shown as like feature: "single (PythonPickle) blob" if the input type is Literal.
After, the input in UI will be shown as like feature: "income" if the input type is Literal.

I think the goal is to support the Literal can be serialized to primitive

[machichima]

What is the goal of a literal type? Is it to provide an enum like behavior or just a weird way of passing primitives

Based on my understanding, Literal type in Python is to set the acceptable value for a variable. Probably a bit like Union but also restrict the value

For example, if def func(a: Literal("a", 1), then we can only call func with func("a") or func(1)

[machichima]

Hi @BarryWu0812 ,

Could you please fix the CI error?

Thanks!

[codecov]

Codecov Report

❌ Patch coverage is 22.44898% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.62%. Comparing base (9effe91) to head (bcd136f).
⚠️ Report is 4 commits behind head on master.

Files with missing lines	Patch %	Lines
flytekit/core/type_engine.py	22.44%	37 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3304      +/-   ##
==========================================
+ Coverage   44.53%   47.62%   +3.09%     
==========================================
  Files         295      215      -80     
  Lines       26792    22634    -4158     
  Branches     2955     2965      +10     
==========================================
- Hits        11931    10779    -1152     
+ Misses      14764    11278    -3486     
- Partials       97      577     +480     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[BarryWu0812]

Hi @BarryWu0812 ,

Could you please fix the CI error?

Thanks!

Hi @machichima , I fixed it, Thanks!

[machichima]

@BarryWu0812

Could you please run an example using Literal as the task argument, and provide a screenshot to ensure the "input" tab in the UI shows correctly instead of single (PythonPickle) blob? As mentioned in issue: #6410

Thanks

[BarryWu0812]

@BarryWu0812

Could you please run an example using Literal as the task argument, and provide a screenshot to ensure the "input" tab in the UI shows correctly instead of single (PythonPickle) blob? As mentioned in issue: #6410

Thanks

Hi @machichima , I have uploaded the result. Thanks

[pingsutw]

Thanks for working on it. Could you also add some unit tests

[pingsutw]

literal_type will always have an annotation attribute, right? I think we can just do if literal_type.annotation and literal_type.annotation.annotations