Hi,
According to [1]:
Empty DataFrames/Series will now default to have a ``RangeIndex``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Before, constructing an empty (where ``data`` is ``None`` or an empty
list-like argument) :class:`Series` or :class:`DataFrame` without
specifying the axes (``index=None``, ``columns=None``) would return the
axes as empty :class:`Index` with object dtype.
Now, the axes return an empty :class:`RangeIndex` (:issue:`49572`).
*Previous behavior*:
.. code-block:: ipython
In [8]: pd.Series().index
Out[8]:
Index([], dtype='object')
In [9] pd.DataFrame().axes
Out[9]:
[Index([], dtype='object'), Index([], dtype='object')]
*New behavior*:
.. ipython:: python
pd.Series().index
pd.DataFrame().axes
empty dataframes are not equals anymore with pandas > 2.0 in this case:
test.py
-------
import pandas as pd
from pandas.testing import assert_frame_equal
df1 = pd.DataFrame(columns=['s1', 's2', 's3']).astype(float)
df2 = pd.DataFrame(columns=['s1', 's2', 's3'], index=[]).astype(float)
try:
assert_frame_equal(df1, df2)
except AssertionError:
raise
print('OK')
Output with pandas 2.1
----------------------
> python3.11 test.py
Traceback (most recent call last):
File "/build/test.py", line 8, in <module>
assert_frame_equal(df1,df2)
File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 1156,
in assert_frame_equal
assert_index_equal(
File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 260,
in assert_index_equal
_check_types(left, right, obj=obj)
File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 236,
in _check_types
assert_attr_equal("inferred_type", left, right, obj=obj)
File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 414,
in assert_attr_equal
raise_assert_detail(obj, msg, left_attr, right_attr)
File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 598,
in raise_assert_detail
raise AssertionError(msg)
AssertionError: DataFrame.index are different
Attribute "inferred_type" are different
[left]: integer
[right]: empty
Output with pandas 1.5
----------------------
> python3.11 test.py
OK
Please note the attached patch makes autopkgtestpass as well with
pandas 1.5.2.
On Sun, 18 Feb 2024 14:45:29 +0100 Andreas Tille <andr...@an3as.eu> wrote:
> Hi again,
>
> Am Sun, Feb 18, 2024 at 12:25:49PM +0100 schrieb Andreas Tille:
> > I just realised that a new qiime version is out. I will upgrade
> > to latest upstream and see how this might affect this issue
>
> The new qiime upstream version does not change anything. After I
> switched q2-* packages to run autopkgtest for `py3versions -s` I
> realised the problem below exist in several of the q2-* packages so its
> rather no Pandas issue but a Python3.12 problem which in parallel to the
> Pandas migration showed up.
>
> If you might have any hint how to deal with these (no matter for the
> old or the new qiime package since I assume the patch will apply
> to both, it would be really appreciated.
>
> Kind regards
> Andreas.
>
> > Am Sun, Feb 18, 2024 at 12:11:04PM +0100 schrieb Andreas Tille:
> > > Control: tags -1 help
> > >
> > > Hi again,
> > >
> > > I hope to approach the last remaining Pandas issue for the qiime
> > > ecosystem. As it has become obvious in the q2-types package I'm now
> > > facing pretty similar errors when running the q2-quality-control
> > > package which can be seen in full length in Salsa-CI[3] and contains
> > > errors like:
> > >
> > > E AttributeError: 'ProvenancePath' object has no attribute '_drv'
> > > E AttributeError: 'ProvenancePath' object has no attribute '_raw_paths'
> > > E AttributeError: 'ProvenancePath' object has no attribute '_str'
> > > E AttributeError: 'OutPath' object has no attribute '_str'
> > >
> > > This all goes back to the qiime package but I admit I have no idea
> > > how to fix this.
> > >
> > > Kind regards
> > > Andreas.
> > >
> > >
> > > [3]
> > > https://salsa.debian.org/med-team/q2-quality-control/-/jobs/5320775#L700
> > >
> > > Am Sat, Feb 17, 2024 at 11:36:41AM +0100 schrieb Andreas Tille:
> > > > Hi,
> > > >
> > > > as reported in a qiime2 issue[1] there is some problem with Python3.12
> > > > in the tests of the q2-* packages which are all using the qiime package.
> > > > This problem is currently hidden from the tests made by Python3.12
> > > > porters but it became obvious now on Salsa CI[2]. I tried to fiddle
> > > > around a bit with the qiime code but with no success at all. Any help
> > > > would be welcome.
> > > >
> > > > Kind regards
> > > > Andreas.
> > > >
> > > > [1] https://github.com/qiime2/qiime2/issues/751
> > > > [2] https://salsa.debian.org/med-team/q2-types/-/jobs/5313640#L900
> > > >
I really don't know how to fix this issue with python 3.12, sorry :(
Kind Regards
[1]
https://sources.debian.org/src/pandas/2.1.4%2Bdfsg-4/doc/source/whatsnew/v2.0.0.rst/#L586
--- a/q2_quality_control/tests/test_quality_control.py
+++ b/q2_quality_control/tests/test_quality_control.py
@@ -486,7 +486,7 @@
def test_evaluate_composition_dont_test_all_levels(self):
empty_expectations = pd.DataFrame(
- columns=['s1', 's2', 's3']).astype(float)
+ columns=['s1', 's2', 's3'], index=[]).astype(float)
empty_expectations.index.name = 'Taxon'
mc = self.misclassified.loc[[
'k__Ag;p__Bg;c__Cg;o__Dg;f__Eg;g__Fg;s__Gg']]
