How to handle a complex observation space?

[luxiaolei]

Hi,

My observation space is rather complicated, such as:

self.observation_space = spaces.Tuple((
    spaces.Dict({
        'k1': spaces.Box(low=0, high=np.inf, shape=(1,), dtype=np.float32),
        'k2': spaces.Box(low=-1, high=1, shape=(1,), dtype=np.float32),
    }), # feed into a FC network 
    spaces.Box(low=-np.inf, high=np.inf, shape=(seq_len, feature_dim), dtype=np.float32), # feed into a transformer net wrok
    spaces.Box(low=0, high=255, shape=(chart_seq_len, chart_seq_len, 3), dtype=np.uint8), # feed into a Conv style net work
    spaces.Box(low=0, high=np.array([np.inf, 3]), shape=(None, 2), dtype=np.float32) # feed into a FC network
))

different part of the observation need to be fed into different encoder(model), may I ask is there any templet code to handle such case?

Thanks in advance.

[PaParaZz1]

You should define your own encoder network and pass it to the pre-defined template model and policy. We will add an example code to in a week.

BTW, which DRL policy do you prefer to use? We can add an example on this policy.

[luxiaolei]

Thx for your reply, yes that would be so nice to have an example!
For policy I use Hybird action space PPO

[zjowowen]

Hi XiaoLei!
We provide an example of how to define a complex observation encoder and to use it in the PPO policy model.
The file is ppo_with_complex_obs.py .
You need to git fetch or pull the latest DI-engine to run this file.

[luxiaolei]

Great news! thanks alot !