Hi, I was applying feature extraction in a time series dataframe, but I found this issue:
import tsfel
cfg = tsfel.get_features_by_domain(["statistical", "temporal", "spectral"])
X_train = tsfel.time_series_features_extractor(cfg, x_train_array, fs=1/86400)
X_test = tsfel.time_series_features_extractor(cfg, x_test_array, fs=1/86400)
But I got this error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[10], line 4
1 import tsfel
2 cfg = tsfel.get_features_by_domain(["statistical", "temporal", "spectral"])
----> 4 X_train = tsfel.time_series_features_extractor(cfg, x_train_array, fs=1/86400)
5 X_test = tsfel.time_series_features_extractor(cfg, x_test_array, fs=1/86400)
File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/tsfel/feature_extraction/calc_features.py:390, in time_series_features_extractor(config, timeseries, fs, window_size, overlap, verbose, **kwargs)
379 features_final = calc_window_features(
380 config,
381 timeseries,
(...) 386 single_window=True,
387 )
389 # Assuring the same feature extraction order
--> 390 features_final = features_final.reindex(sorted(features_final.columns), axis=1)
391 return features_final.reset_index(drop=True)
File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/frame.py:6078, in DataFrame.reindex(self, labels, index, columns, axis, method, copy, level, fill_value, limit, tolerance)
5843 def reindex(
5844 self,
5845 labels=None,
(...) 5855 tolerance=None,
5856 ) -> DataFrame:
5857 """
5858 Conform DataFrame to new index with optional filling logic.
5859
(...) 6076 See the :ref:`user guide <basics.reindexing>` for more.
6077 """
-> 6078 return super().reindex(
6079 labels=labels,
6080 index=index,
6081 columns=columns,
6082 axis=axis,
6083 method=method,
6084 level=level,
6085 fill_value=fill_value,
6086 limit=limit,
6087 tolerance=tolerance,
6088 copy=copy,
6089 )
File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/generic.py:5476, in NDFrame.reindex(self, labels, index, columns, axis, method, copy, level, fill_value, limit, tolerance)
5473 return self._reindex_multi(axes, fill_value)
5475 # perform the reindex on the axes
-> 5476 return self._reindex_axes(
5477 axes, level, limit, tolerance, method, fill_value
5478 ).__finalize__(self, method="reindex")
File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/generic.py:5498, in NDFrame._reindex_axes(self, axes, level, limit, tolerance, method, fill_value)
5495 continue
5497 ax = self._get_axis(a)
-> 5498 new_index, indexer = ax.reindex(
5499 labels, level=level, limit=limit, tolerance=tolerance, method=method
5500 )
5502 axis = self._get_axis_number(a)
5503 obj = obj._reindex_with_indexers(
5504 {axis: [new_index, indexer]},
5505 fill_value=fill_value,
5506 allow_dups=False,
5507 )
File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/indexes/base.py:4253, in Index.reindex(self, target, method, level, limit, tolerance)
4250 raise ValueError("cannot handle a non-unique multi-index!")
4251 elif not self.is_unique:
4252 # GH#42568
-> 4253 raise ValueError("cannot reindex on an axis with duplicate labels")
4254 else:
4255 indexer, _ = self.get_indexer_non_unique(target)
ValueError: cannot reindex on an axis with duplicate labels
It seems like internally is creating a columns dataframe that already exists with that name, so, at the moment to apply reindex the error raises.
When I don't specify a fs (use the default that until I know is 100 Hz), this error doesn't raise.
Maybe it can't work with non integer sample frequency.
Steps to recreate
I was using a public dataset, so I don't have problem to give the exact data that I was using.
- Download .parquet file
- Read the file in a dataframe (pandas or polars)
- Use the following code:
# Series to list of numpy arrays
x_train_array = []
for array in x_train[:, 2:].to_numpy():
x_train_array.append(array)
import tsfel
cfg = tsfel.get_features_by_domain(["statistical", "temporal", "spectral"])
X_train = tsfel.time_series_features_extractor(cfg, x_train_array, fs=1/86400)
I know that the problem is with spectral feature, cause a run an experiment individually with each feature:
Data: https://drive.google.com/file/d/1sTXC_SCIWi-oKoYOC5QxBN9ZlFl_ha7W/view?usp=sharing
Version: tsfel == 0.2.0
Hi, I was applying feature extraction in a time series dataframe, but I found this issue:
But I got this error:
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[10], line 4 1 import tsfel 2 cfg = tsfel.get_features_by_domain(["statistical", "temporal", "spectral"]) ----> 4 X_train = tsfel.time_series_features_extractor(cfg, x_train_array, fs=1/86400) 5 X_test = tsfel.time_series_features_extractor(cfg, x_test_array, fs=1/86400) File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/tsfel/feature_extraction/calc_features.py:390, in time_series_features_extractor(config, timeseries, fs, window_size, overlap, verbose, **kwargs) 379 features_final = calc_window_features( 380 config, 381 timeseries, (...) 386 single_window=True, 387 ) 389 # Assuring the same feature extraction order --> 390 features_final = features_final.reindex(sorted(features_final.columns), axis=1) 391 return features_final.reset_index(drop=True) File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/frame.py:6078, in DataFrame.reindex(self, labels, index, columns, axis, method, copy, level, fill_value, limit, tolerance) 5843 def reindex( 5844 self, 5845 labels=None, (...) 5855 tolerance=None, 5856 ) -> DataFrame: 5857 """ 5858 Conform DataFrame to new index with optional filling logic. 5859 (...) 6076 See the :ref:`user guide <basics.reindexing>` for more. 6077 """ -> 6078 return super().reindex( 6079 labels=labels, 6080 index=index, 6081 columns=columns, 6082 axis=axis, 6083 method=method, 6084 level=level, 6085 fill_value=fill_value, 6086 limit=limit, 6087 tolerance=tolerance, 6088 copy=copy, 6089 ) File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/generic.py:5476, in NDFrame.reindex(self, labels, index, columns, axis, method, copy, level, fill_value, limit, tolerance) 5473 return self._reindex_multi(axes, fill_value) 5475 # perform the reindex on the axes -> 5476 return self._reindex_axes( 5477 axes, level, limit, tolerance, method, fill_value 5478 ).__finalize__(self, method="reindex") File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/generic.py:5498, in NDFrame._reindex_axes(self, axes, level, limit, tolerance, method, fill_value) 5495 continue 5497 ax = self._get_axis(a) -> 5498 new_index, indexer = ax.reindex( 5499 labels, level=level, limit=limit, tolerance=tolerance, method=method 5500 ) 5502 axis = self._get_axis_number(a) 5503 obj = obj._reindex_with_indexers( 5504 {axis: [new_index, indexer]}, 5505 fill_value=fill_value, 5506 allow_dups=False, 5507 ) File ~/Documentos/Dev/Startup/poc-data-china-energy/.venv/lib/python3.13/site-packages/pandas/core/indexes/base.py:4253, in Index.reindex(self, target, method, level, limit, tolerance) 4250 raise ValueError("cannot handle a non-unique multi-index!") 4251 elif not self.is_unique: 4252 # GH#42568 -> 4253 raise ValueError("cannot reindex on an axis with duplicate labels") 4254 else: 4255 indexer, _ = self.get_indexer_non_unique(target) ValueError: cannot reindex on an axis with duplicate labelsIt seems like internally is creating a columns dataframe that already exists with that name, so, at the moment to apply reindex the error raises.
When I don't specify a
fs(use the default that until I know is 100 Hz), this error doesn't raise.Maybe it can't work with non integer sample frequency.
Steps to recreate
I was using a public dataset, so I don't have problem to give the exact data that I was using.
I know that the problem is with spectral feature, cause a run an experiment individually with each feature:
Data: https://drive.google.com/file/d/1sTXC_SCIWi-oKoYOC5QxBN9ZlFl_ha7W/view?usp=sharing
Version:
tsfel == 0.2.0