Compare text

Find the difference between two text files

Real-time editor

Hide unchanged lines

Disable line wrap

Layout

Diff precision

Syntax highlighting

Diffchecker Desktop The most secure way to run Diffchecker. Get the Diffchecker Desktop app: your diffs never leave your computer!Get Desktop

Build modules: pyspark-sql, pyspark-mllib, pyspark-resource

Created 5 years agoDiff never expires

145 removals

Lines
Total
Removed

Characters
Total
Removed

To continue using this feature, upgrade to Diffchecker Pro View Pricing

401 lines

144 additions

Lines
Total
Added

Characters
Total
Added

To continue using this feature, upgrade to Diffchecker Pro View Pricing

402 lines

Before

After

https://github.com/Yikun/spark/runs/2959151757?check_suite_focus=true

https://github.com/Yikun/spark/runs/2968285996

[success] Total time: 206 s (03:26), completed Jul 2, 2021 1:54:02 AM

========================================================================

Running Spark unit tests

========================================================================

Running PySpark tests

========================================================================

'PYARROW_IGNORE_TIMEZONE' environment variable was not set. It is required to set this environment variable to '1' in both driver and executor sides if you use pyarrow>=2.0.0. pandas-on-Spark will set it for you but it does not work if there is a Spark context already launched.

Running PySpark tests. Output is in /__w/spark/spark/python/unit-tests.log

Will test against the following Python executables: ['python3.6', 'python3.9', 'pypy3']

Will test the following Python modules: ['pyspark-sql', 'pyspark-resource', 'pyspark-mllib']

python3.6 python_implementation is CPython

python3.6 version is: Python 3.6.13

python3.9 python_implementation is CPython

python3.9 version is: Python 3.9.5

pypy3 python_implementation is PyPy

pypy3 version is: Python 3.7.10 (7.3.5+dfsg-1~ppa1~ubuntu20.04, May 23 2021, 14:57:07)

[PyPy 7.3.5 with GCC 9.3.0]

Starting test(pypy3): pyspark.sql.tests.test_arrow

Starting test(pypy3): pyspark.sql.tests.test_catalog

Starting test(pypy3): pyspark.sql.tests.test_arrow

Finished test(pypy3): pyspark.sql.tests.test_arrow (1s) ... 69 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_column

Finished test(pypy3): pyspark.sql.tests.test_catalog (24s)

Finished test(pypy3): pyspark.sql.tests.test_catalog (21s)

Starting test(pypy3): pyspark.sql.tests.test_conf

Finished test(pypy3): pyspark.sql.tests.test_column (28s)

Finished test(pypy3): pyspark.sql.tests.test_column (27s)

Starting test(pypy3): pyspark.sql.tests.test_context

Finished test(pypy3): pyspark.sql.tests.test_conf (15s)

Finished test(pypy3): pyspark.sql.tests.test_conf (14s)

Starting test(pypy3): pyspark.sql.tests.test_dataframe

Finished test(pypy3): pyspark.sql.tests.test_context (26s)

Starting test(pypy3): pyspark.sql.tests.test_datasources

Finished test(pypy3): pyspark.sql.tests.test_datasources (37s)

Finished test(pypy3): pyspark.sql.tests.test_datasources (34s)

Starting test(pypy3): pyspark.sql.tests.test_functions

Finished test(pypy3): pyspark.sql.tests.test_dataframe (63s) ... 4 tests were skipped

Finished test(pypy3): pyspark.sql.tests.test_dataframe (58s) ... 4 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_group

Finished test(pypy3): pyspark.sql.tests.test_group (27s)

Finished test(pypy3): pyspark.sql.tests.test_group (25s)

Starting test(pypy3): pyspark.sql.tests.test_pandas_cogrouped_map

Finished test(pypy3): pyspark.sql.tests.test_pandas_cogrouped_map (1s) ... 16 tests were skipped

Finished test(pypy3): pyspark.sql.tests.test_pandas_cogrouped_map (2s) ... 16 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_pandas_grouped_map

Finished test(pypy3): pyspark.sql.tests.test_pandas_grouped_map (2s) ... 21 tests were skipped

Finished test(pypy3): pyspark.sql.tests.test_pandas_grouped_map (1s) ... 21 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_pandas_map

Finished test(pypy3): pyspark.sql.tests.test_pandas_map (2s) ... 7 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_pandas_udf

Finished test(pypy3): pyspark.sql.tests.test_pandas_udf (2s) ... 6 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_grouped_agg

Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_grouped_agg (2s) ... 13 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_scalar

Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_scalar (2s) ... 51 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_typehints

Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_typehints (2s) ... 10 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_window

Finished test(pypy3): pyspark.sql.tests.test_functions (53s)

Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_window (2s) ... 14 tests were skipped

Starting test(pypy3): pyspark.sql.tests.test_readwriter

Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_window (1s) ... 14 tests were skipped

Finished test(pypy3): pyspark.sql.tests.test_functions (49s)

Starting test(pypy3): pyspark.sql.tests.test_serde

Finished test(pypy3): pyspark.sql.tests.test_serde (34s)

Finished test(pypy3): pyspark.sql.tests.test_serde (32s)

Starting test(pypy3): pyspark.sql.tests.test_session

Finished test(pypy3): pyspark.sql.tests.test_readwriter (45s)

Finished test(pypy3): pyspark.sql.tests.test_readwriter (38s)

Starting test(pypy3): pyspark.sql.tests.test_streaming

Finished test(pypy3): pyspark.sql.tests.test_session (40s)

Finished test(pypy3): pyspark.sql.tests.test_session (36s)

Starting test(pypy3): pyspark.sql.tests.test_types

Finished test(pypy3): pyspark.sql.tests.test_streaming (51s)

Finished test(pypy3): pyspark.sql.tests.test_streaming (46s)

Starting test(pypy3): pyspark.sql.tests.test_udf

Finished test(pypy3): pyspark.sql.tests.test_types (65s)

Finished test(pypy3): pyspark.sql.tests.test_types (71s)

Starting test(pypy3): pyspark.sql.tests.test_utils

Finished test(pypy3): pyspark.sql.tests.test_utils (22s)

Finished test(pypy3): pyspark.sql.tests.test_udf (71s)

Starting test(python3.6): pyspark.mllib.tests.test_algorithms

Finished test(pypy3): pyspark.sql.tests.test_udf (67s)

Finished test(pypy3): pyspark.sql.tests.test_utils (18s)

Starting test(python3.6): pyspark.mllib.tests.test_feature

Finished test(python3.6): pyspark.mllib.tests.test_feature (43s)

Finished test(python3.6): pyspark.mllib.tests.test_feature (38s)

Starting test(python3.6): pyspark.mllib.tests.test_linalg

Finished test(python3.6): pyspark.mllib.tests.test_algorithms (91s)

Finished test(python3.6): pyspark.mllib.tests.test_algorithms (85s)

Starting test(python3.6): pyspark.mllib.tests.test_stat

Finished test(python3.6): pyspark.mllib.tests.test_stat (31s)

Finished test(python3.6): pyspark.mllib.tests.test_stat (29s)

Starting test(python3.6): pyspark.mllib.tests.test_streaming_algorithms

Finished test(python3.6): pyspark.mllib.tests.test_linalg (83s)

Finished test(python3.6): pyspark.mllib.tests.test_linalg (77s)

Starting test(python3.6): pyspark.mllib.tests.test_util

Finished test(python3.6): pyspark.mllib.tests.test_util (14s)

Starting test(python3.6): pyspark.sql.tests.test_arrow

Finished test(python3.6): pyspark.sql.tests.test_arrow (49s)

Finished test(python3.6): pyspark.sql.tests.test_arrow (51s)

Starting test(python3.6): pyspark.sql.tests.test_catalog

Finished test(python3.6): pyspark.sql.tests.test_catalog (26s)

Finished test(python3.6): pyspark.sql.tests.test_catalog (27s)

Starting test(python3.6): pyspark.sql.tests.test_column

Finished test(python3.6): pyspark.sql.tests.test_column (27s)

Starting test(python3.6): pyspark.sql.tests.test_conf

Finished test(python3.6): pyspark.sql.tests.test_conf (15s)

Finished test(python3.6): pyspark.sql.tests.test_conf (13s)

Starting test(python3.6): pyspark.sql.tests.test_context

Finished test(python3.6): pyspark.sql.tests.test_context (20s)

Finished test(python3.6): pyspark.sql.tests.test_context (19s)

Starting test(python3.6): pyspark.sql.tests.test_dataframe

Finished test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (187s)

Finished test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (190s)

Starting test(python3.6): pyspark.sql.tests.test_datasources

Finished test(python3.6): pyspark.sql.tests.test_datasources (31s)

Finished test(python3.6): pyspark.sql.tests.test_dataframe (59s) ... 2 tests were skipped

Starting test(python3.6): pyspark.sql.tests.test_functions

Finished test(python3.6): pyspark.sql.tests.test_dataframe (63s) ... 2 tests were skipped

Finished test(python3.6): pyspark.sql.tests.test_datasources (28s)

Starting test(python3.6): pyspark.sql.tests.test_group

Finished test(python3.6): pyspark.sql.tests.test_group (24s)

Finished test(python3.6): pyspark.sql.tests.test_group (21s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map

Finished test(python3.6): pyspark.sql.tests.test_functions (48s)

Finished test(python3.6): pyspark.sql.tests.test_functions (45s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_grouped_map

Finished test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (46s)

Finished test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (43s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_map

Finished test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (52s)

Finished test(python3.6): pyspark.sql.tests.test_pandas_map (34s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_udf

Finished test(python3.6): pyspark.sql.tests.test_pandas_map (33s)

Finished test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (55s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg

Finished test(python3.6): pyspark.sql.tests.test_pandas_udf (31s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar

Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (70s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints

Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints (33s)

Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_window

Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (112s)

Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (105s)

Starting test(python3.6): pyspark.sql.tests.test_readwriter

Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (50s)

Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (48s)

Starting test(python3.6): pyspark.sql.tests.test_serde

Finished test(python3.6): pyspark.sql.tests.test_readwriter (44s)

Finished test(python3.6): pyspark.sql.tests.test_readwriter (40s)

Starting test(python3.6): pyspark.sql.tests.test_session

Finished test(python3.6): pyspark.sql.tests.test_serde (33s)

Starting test(python3.6): pyspark.sql.tests.test_streaming

Finished test(python3.6): pyspark.sql.tests.test_session (38s)

Finished test(python3.6): pyspark.sql.tests.test_session (36s)

Starting test(python3.6): pyspark.sql.tests.test_types

Finished test(python3.6): pyspark.sql.tests.test_streaming (45s)

Starting test(python3.6): pyspark.sql.tests.test_udf

Finished test(python3.6): pyspark.sql.tests.test_types (53s)

Finished test(python3.6): pyspark.sql.tests.test_types (52s)

Starting test(python3.6): pyspark.sql.tests.test_utils

Finished test(python3.6): pyspark.sql.tests.test_utils (17s)

Finished test(python3.6): pyspark.sql.tests.test_utils (15s)

Starting test(python3.9): pyspark.mllib.tests.test_algorithms

Finished test(python3.6): pyspark.sql.tests.test_udf (58s)

Finished test(python3.6): pyspark.sql.tests.test_udf (56s)

Starting test(python3.9): pyspark.mllib.tests.test_feature

Finished test(python3.9): pyspark.mllib.tests.test_feature (41s)

Finished test(python3.9): pyspark.mllib.tests.test_feature (40s)

Starting test(python3.9): pyspark.mllib.tests.test_linalg

Finished test(python3.9): pyspark.mllib.tests.test_algorithms (91s)

Finished test(python3.9): pyspark.mllib.tests.test_algorithms (82s)

Starting test(python3.9): pyspark.mllib.tests.test_stat

Finished test(python3.9): pyspark.mllib.tests.test_stat (30s)

Finished test(python3.9): pyspark.mllib.tests.test_stat (31s)

Starting test(python3.9): pyspark.mllib.tests.test_streaming_algorithms

Finished test(python3.9): pyspark.mllib.tests.test_linalg (81s)

Finished test(python3.9): pyspark.mllib.tests.test_linalg (77s)

Starting test(python3.9): pyspark.mllib.tests.test_util

Finished test(python3.9): pyspark.mllib.tests.test_util (15s)

Finished test(python3.9): pyspark.mllib.tests.test_util (11s)

Starting test(python3.9): pyspark.sql.tests.test_arrow

Finished test(python3.9): pyspark.sql.tests.test_arrow (46s)

Finished test(python3.9): pyspark.sql.tests.test_arrow (47s)

Starting test(python3.9): pyspark.sql.tests.test_catalog

Finished test(python3.9): pyspark.sql.tests.test_catalog (25s)

Finished test(python3.9): pyspark.sql.tests.test_catalog (20s)

Starting test(python3.9): pyspark.sql.tests.test_column

Finished test(python3.9): pyspark.sql.tests.test_column (27s)

Finished test(python3.9): pyspark.sql.tests.test_column (23s)

Starting test(python3.9): pyspark.sql.tests.test_conf

Finished test(python3.9): pyspark.sql.tests.test_conf (14s)

Finished test(python3.9): pyspark.sql.tests.test_conf (12s)

Starting test(python3.9): pyspark.sql.tests.test_context

Finished test(python3.9): pyspark.sql.tests.test_context (20s)

Finished test(python3.9): pyspark.sql.tests.test_context (18s)

Starting test(python3.9): pyspark.sql.tests.test_dataframe

Finished test(python3.9): pyspark.mllib.tests.test_streaming_algorithms (181s)

Finished test(python3.9): pyspark.mllib.tests.test_streaming_algorithms (172s)

Starting test(python3.9): pyspark.sql.tests.test_datasources

Finished test(python3.9): pyspark.sql.tests.test_datasources (28s)

Finished test(python3.9): pyspark.sql.tests.test_datasources (29s)

Starting test(python3.9): pyspark.sql.tests.test_functions

Finished test(python3.9): pyspark.sql.tests.test_dataframe (59s) ... 2 tests were skipped

Finished test(python3.9): pyspark.sql.tests.test_dataframe (58s) ... 2 tests were skipped

Starting test(python3.9): pyspark.sql.tests.test_group

Finished test(python3.9): pyspark.sql.tests.test_group (21s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_cogrouped_map

Finished test(python3.9): pyspark.sql.tests.test_functions (48s)

Finished test(python3.9): pyspark.sql.tests.test_functions (45s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_grouped_map

Finished test(python3.9): pyspark.sql.tests.test_pandas_cogrouped_map (46s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_cogrouped_map (40s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_map

Finished test(python3.9): pyspark.sql.tests.test_pandas_grouped_map (52s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_map (29s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_udf

Finished test(python3.9): pyspark.sql.tests.test_pandas_map (33s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_grouped_map (49s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_grouped_agg

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf (31s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf (25s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_scalar

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_grouped_agg (67s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_grouped_agg (63s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_typehints

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_typehints (36s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_typehints (29s)

Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_window

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_scalar (107s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_scalar (98s)

Starting test(python3.9): pyspark.sql.tests.test_readwriter

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_window (52s)

Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_window (46s)

Starting test(python3.9): pyspark.sql.tests.test_serde

Finished test(python3.9): pyspark.sql.tests.test_readwriter (43s)

Finished test(python3.9): pyspark.sql.tests.test_readwriter (44s)

Starting test(python3.9): pyspark.sql.tests.test_session

Finished test(python3.9): pyspark.sql.tests.test_serde (33s)

Finished test(python3.9): pyspark.sql.tests.test_serde (34s)

Starting test(python3.9): pyspark.sql.tests.test_streaming

Finished test(python3.9): pyspark.sql.tests.test_session (37s)

Starting test(python3.9): pyspark.sql.tests.test_types

Finished test(python3.9): pyspark.sql.tests.test_streaming (45s)

Finished test(python3.9): pyspark.sql.tests.test_streaming (46s)

Starting test(python3.9): pyspark.sql.tests.test_udf

Finished test(python3.9): pyspark.sql.tests.test_types (53s)

Starting test(python3.9): pyspark.sql.tests.test_utils

Finished test(python3.9): pyspark.sql.tests.test_utils (16s)

Starting test(pypy3): pyspark.resource.tests.test_resources

Finished test(python3.9): pyspark.sql.tests.test_udf (56s)

Finished test(python3.9): pyspark.sql.tests.test_udf (58s)

Starting test(pypy3): pyspark.sql.avro.functions

Finished test(pypy3): pyspark.resource.tests.test_resources (8s)

Finished test(pypy3): pyspark.resource.tests.test_resources (7s)

Starting test(pypy3): pyspark.sql.catalog

Finished test(pypy3): pyspark.sql.avro.functions (19s)

Finished test(pypy3): pyspark.sql.avro.functions (20s)

Starting test(pypy3): pyspark.sql.column

Finished test(pypy3): pyspark.sql.catalog (19s)

Finished test(pypy3): pyspark.sql.catalog (20s)

Starting test(pypy3): pyspark.sql.conf

Finished test(pypy3): pyspark.sql.conf (8s)

Starting test(pypy3): pyspark.sql.context

Finished test(pypy3): pyspark.sql.column (36s)

Starting test(pypy3): pyspark.sql.dataframe

Finished test(pypy3): pyspark.sql.context (32s)

Starting test(pypy3): pyspark.sql.functions

Finished test(pypy3): pyspark.sql.dataframe (71s)

Finished test(pypy3): pyspark.sql.dataframe (73s)

Starting test(pypy3): pyspark.sql.group

Finished test(pypy3): pyspark.sql.functions (88s)

Starting test(pypy3): pyspark.sql.pandas.conversion

Finished test(pypy3): pyspark.sql.group (36s)

Finished test(pypy3): pyspark.sql.group (33s)

Starting test(pypy3): pyspark.sql.pandas.group_ops

Finished test(pypy3): pyspark.sql.pandas.conversion (10s)

Finished test(pypy3): pyspark.sql.pandas.conversion (11s)

Starting test(pypy3): pyspark.sql.pandas.map_ops

Finished test(pypy3): pyspark.sql.pandas.group_ops (13s)

Finished test(pypy3): pyspark.sql.pandas.group_ops (12s)

Starting test(pypy3): pyspark.sql.pandas.serializers

Finished test(pypy3): pyspark.sql.pandas.serializers (0s)

Starting test(pypy3): pyspark.sql.pandas.typehints

Finished test(pypy3): pyspark.sql.pandas.map_ops (12s)

Starting test(pypy3): pyspark.sql.pandas.types

Finished test(pypy3): pyspark.sql.pandas.typehints (0s)

Starting test(pypy3): pyspark.sql.pandas.types

Starting test(pypy3): pyspark.sql.pandas.utils

Finished test(pypy3): pyspark.sql.pandas.types (0s)

Starting test(pypy3): pyspark.sql.pandas.utils

Finished test(pypy3): pyspark.sql.pandas.map_ops (12s)

Starting test(pypy3): pyspark.sql.readwriter

Finished test(pypy3): pyspark.sql.pandas.utils (0s)

Starting test(pypy3): pyspark.sql.session

Finished test(pypy3): pyspark.sql.session (27s)

Finished test(pypy3): pyspark.sql.session (26s)

Starting test(pypy3): pyspark.sql.streaming

Finished test(pypy3): pyspark.sql.readwriter (31s)

Finished test(pypy3): pyspark.sql.readwriter (29s)

Starting test(pypy3): pyspark.sql.types

Finished test(pypy3): pyspark.sql.types (10s)

Finished test(pypy3): pyspark.sql.types (9s)

Starting test(pypy3): pyspark.sql.udf

Finished test(pypy3): pyspark.sql.streaming (19s)

Finished test(pypy3): pyspark.sql.streaming (15s)

Starting test(pypy3): pyspark.sql.window

Finished test(pypy3): pyspark.sql.udf (18s)

Finished test(pypy3): pyspark.sql.udf (17s)

Starting test(python3.6): pyspark.mllib.classification

Finished test(pypy3): pyspark.sql.window (20s)

Finished test(pypy3): pyspark.sql.window (18s)

Starting test(python3.6): pyspark.mllib.clustering

Finished test(python3.6): pyspark.mllib.classification (56s)

Starting test(python3.6): pyspark.mllib.evaluation

Finished test(python3.6): pyspark.mllib.clustering (73s)

Finished test(python3.6): pyspark.mllib.clustering (67s)

Starting test(python3.6): pyspark.mllib.feature

Finished test(python3.6): pyspark.mllib.evaluation (34s)

Finished test(python3.6): pyspark.mllib.evaluation (26s)

Starting test(python3.6): pyspark.mllib.fpm

Finished test(python3.6): pyspark.mllib.fpm (28s)

Finished test(python3.6): pyspark.mllib.feature (32s)

Starting test(python3.6): pyspark.mllib.linalg.__init__

Finished test(python3.6): pyspark.mllib.linalg.__init__ (0s)

Starting test(python3.6): pyspark.mllib.linalg.distributed

Finished test(python3.6): pyspark.mllib.feature (38s)

Finished test(python3.6): pyspark.mllib.fpm (24s)

Starting test(python3.6): pyspark.mllib.random

Finished test(python3.6): pyspark.mllib.random (15s)

Finished test(python3.6): pyspark.mllib.random (13s)

Starting test(python3.6): pyspark.mllib.recommendation

Finished test(python3.6): pyspark.mllib.recommendation (61s)

Finished test(python3.6): pyspark.mllib.linalg.distributed (71s)

Starting test(python3.6): pyspark.mllib.regression

Finished test(python3.6): pyspark.mllib.linalg.distributed (79s)

Finished test(python3.6): pyspark.mllib.recommendation (56s)

Starting test(python3.6): pyspark.mllib.stat.KernelDensity

Finished test(python3.6): pyspark.mllib.stat.KernelDensity (0s)

Starting test(python3.6): pyspark.mllib.stat._statistics

Finished test(python3.6): pyspark.mllib.stat._statistics (26s)

Finished test(python3.6): pyspark.mllib.stat._statistics (22s)

Starting test(python3.6): pyspark.mllib.tree

Finished test(python3.6): pyspark.mllib.regression (41s)

Finished test(python3.6): pyspark.mllib.regression (35s)

Starting test(python3.6): pyspark.mllib.util

Finished test(python3.6): pyspark.mllib.tree (27s)

Finished test(python3.6): pyspark.mllib.tree (24s)

Starting test(python3.6): pyspark.resource.tests.test_resources

Finished test(python3.6): pyspark.resource.tests.test_resources (8s)

Finished test(python3.6): pyspark.resource.tests.test_resources (7s)

Starting test(python3.6): pyspark.sql.avro.functions

Finished test(python3.6): pyspark.mllib.util (28s)

Finished test(python3.6): pyspark.mllib.util (25s)

Starting test(python3.6): pyspark.sql.catalog

Finished test(python3.6): pyspark.sql.avro.functions (19s)

Starting test(python3.6): pyspark.sql.column

Finished test(python3.6): pyspark.sql.catalog (17s)

Finished test(python3.6): pyspark.sql.catalog (16s)

Starting test(python3.6): pyspark.sql.conf

Finished test(python3.6): pyspark.sql.conf (8s)

Starting test(python3.6): pyspark.sql.context

Finished test(python3.6): pyspark.sql.column (32s)

Finished test(python3.6): pyspark.sql.column (33s)

Starting test(python3.6): pyspark.sql.dataframe

Finished test(python3.6): pyspark.sql.context (28s)

Finished test(python3.6): pyspark.sql.context (26s)

Starting test(python3.6): pyspark.sql.functions

Finished test(python3.6): pyspark.sql.dataframe (64s)

Finished test(python3.6): pyspark.sql.dataframe (57s)

Starting test(python3.6): pyspark.sql.group

Finished test(python3.6): pyspark.sql.functions (73s)

Finished test(python3.6): pyspark.sql.functions (69s)

Starting test(python3.6): pyspark.sql.pandas.conversion

Finished test(python3.6): pyspark.sql.pandas.conversion (10s)

Finished test(python3.6): pyspark.sql.group (27s)

Starting test(python3.6): pyspark.sql.pandas.group_ops

Finished test(python3.6): pyspark.sql.group (28s)

Finished test(python3.6): pyspark.sql.pandas.conversion (9s)

Starting test(python3.6): pyspark.sql.pandas.map_ops

Finished test(python3.6): pyspark.sql.pandas.group_ops (12s)

Finished test(python3.6): pyspark.sql.pandas.group_ops (11s)

Starting test(python3.6): pyspark.sql.pandas.serializers

Finished test(python3.6): pyspark.sql.pandas.map_ops (11s)

Starting test(python3.6): pyspark.sql.pandas.typehints

Finished test(python3.6): pyspark.sql.pandas.serializers (0s)

Starting test(python3.6): pyspark.sql.pandas.typehints

Starting test(python3.6): pyspark.sql.pandas.types

Finished test(python3.6): pyspark.sql.pandas.typehints (0s)

Starting test(python3.6): pyspark.sql.pandas.types

Starting test(python3.6): pyspark.sql.pandas.utils

Finished test(python3.6): pyspark.sql.pandas.types (0s)

Starting test(python3.6): pyspark.sql.pandas.utils

Starting test(python3.6): pyspark.sql.readwriter

Finished test(python3.6): pyspark.sql.pandas.utils (0s)

Starting test(python3.6): pyspark.sql.readwriter

Finished test(python3.6): pyspark.sql.pandas.map_ops (12s)

Starting test(python3.6): pyspark.sql.session

Finished test(python3.6): pyspark.sql.session (25s)

Finished test(python3.6): pyspark.sql.session (23s)

Starting test(python3.6): pyspark.sql.streaming

Finished test(python3.6): pyspark.sql.readwriter (31s)

Finished test(python3.6): pyspark.sql.readwriter (28s)

Starting test(python3.6): pyspark.sql.types

Finished test(python3.6): pyspark.sql.types (10s)

Finished test(python3.6): pyspark.sql.types (9s)

Starting test(python3.6): pyspark.sql.udf

Finished test(python3.6): pyspark.sql.streaming (19s)

Finished test(python3.6): pyspark.sql.streaming (18s)

Starting test(python3.6): pyspark.sql.window

Finished test(python3.6): pyspark.sql.udf (19s)

Finished test(python3.6): pyspark.sql.udf (18s)

Starting test(python3.9): pyspark.mllib.classification

Finished test(python3.6): pyspark.sql.window (20s)

Finished test(python3.6): pyspark.sql.window (19s)

Starting test(python3.9): pyspark.mllib.clustering

Finished test(python3.9): pyspark.mllib.classification (59s)

Starting test(python3.9): pyspark.mllib.evaluation

Finished test(python3.9): pyspark.mllib.clustering (72s)

Starting test(python3.9): pyspark.mllib.feature

Finished test(python3.9): pyspark.mllib.evaluation (30s)

Finished test(python3.9): pyspark.mllib.evaluation (31s)

Starting test(python3.9): pyspark.mllib.fpm

Finished test(python3.9): pyspark.mllib.feature (38s)

Finished test(python3.9): pyspark.mllib.feature (36s)

Starting test(python3.9): pyspark.mllib.linalg.__init__

Finished test(python3.9): pyspark.mllib.linalg.__init__ (0s)

Starting test(python3.9): pyspark.mllib.linalg.distributed

Finished test(python3.9): pyspark.mllib.fpm (30s)

Finished test(python3.9): pyspark.mllib.fpm (28s)

Starting test(python3.9): pyspark.mllib.random

Finished test(python3.9): pyspark.mllib.random (15s)

Finished test(python3.9): pyspark.mllib.random (16s)

Starting test(python3.9): pyspark.mllib.recommendation

Finished test(python3.9): pyspark.mllib.linalg.distributed (78s)

Finished test(python3.9): pyspark.mllib.linalg.distributed (77s)

Starting test(python3.9): pyspark.mllib.regression

Finished test(python3.9): pyspark.mllib.recommendation (59s)

Starting test(python3.9): pyspark.mllib.stat.KernelDensity

Finished test(python3.9): pyspark.mllib.stat.KernelDensity (0s)

Starting test(python3.9): pyspark.mllib.stat._statistics

Finished test(python3.9): pyspark.mllib.stat._statistics (25s)

Finished test(python3.9): pyspark.mllib.stat._statistics (24s)

Starting test(python3.9): pyspark.mllib.tree

Finished test(python3.9): pyspark.mllib.regression (41s)

Finished test(python3.9): pyspark.mllib.regression (38s)

Starting test(python3.9): pyspark.mllib.util

Finished test(python3.9): pyspark.mllib.tree (28s)

Finished test(python3.9): pyspark.mllib.tree (24s)

Starting test(python3.9): pyspark.resource.tests.test_resources

Finished test(python3.9): pyspark.resource.tests.test_resources (8s)

Starting test(python3.9): pyspark.sql.avro.functions

Finished test(python3.9): pyspark.mllib.util (27s)

Finished test(python3.9): pyspark.mllib.util (25s)

Starting test(python3.9): pyspark.sql.catalog

Finished test(python3.9): pyspark.sql.avro.functions (19s)

Finished test(python3.9): pyspark.sql.avro.functions (16s)

Starting test(python3.9): pyspark.sql.column

Finished test(python3.9): pyspark.sql.catalog (17s)

Finished test(python3.9): pyspark.sql.catalog (16s)

Starting test(python3.9): pyspark.sql.conf

Finished test(python3.9): pyspark.sql.conf (8s)

Starting test(python3.9): pyspark.sql.context

Finished test(python3.9): pyspark.sql.column (33s)

Finished test(python3.9): pyspark.sql.column (30s)

Starting test(python3.9): pyspark.sql.dataframe

Finished test(python3.9): pyspark.sql.context (28s)

Finished test(python3.9): pyspark.sql.context (26s)

Starting test(python3.9): pyspark.sql.functions

Finished test(python3.9): pyspark.sql.dataframe (63s)

Finished test(python3.9): pyspark.sql.dataframe (57s)

Starting test(python3.9): pyspark.sql.group

Finished test(python3.9): pyspark.sql.functions (79s)

Finished test(python3.9): pyspark.sql.functions (73s)

Starting test(python3.9): pyspark.sql.pandas.conversion

Finished test(python3.9): pyspark.sql.group (30s)

Finished test(python3.9): pyspark.sql.group (28s)

Starting test(python3.9): pyspark.sql.pandas.group_ops

Finished test(python3.9): pyspark.sql.pandas.conversion (9s)

Starting test(python3.9): pyspark.sql.pandas.map_ops

Finished test(python3.9): pyspark.sql.pandas.group_ops (12s)

Finished test(python3.9): pyspark.sql.pandas.group_ops (11s)

Starting test(python3.9): pyspark.sql.pandas.serializers

Finished test(python3.9): pyspark.sql.pandas.serializers (0s)

Starting test(python3.9): pyspark.sql.pandas.typehints

Finished test(python3.9): pyspark.sql.pandas.typehints (0s)

Starting test(python3.9): pyspark.sql.pandas.types

Finished test(python3.9): pyspark.sql.pandas.map_ops (11s)

Finished test(python3.9): pyspark.sql.pandas.map_ops (10s)

Starting test(python3.9): pyspark.sql.pandas.utils

Finished test(python3.9): pyspark.sql.pandas.types (0s)

Starting test(python3.9): pyspark.sql.readwriter

Finished test(python3.9): pyspark.sql.pandas.utils (0s)

Starting test(python3.9): pyspark.sql.session

Finished test(python3.9): pyspark.sql.session (25s)

Finished test(python3.9): pyspark.sql.session (23s)

Starting test(python3.9): pyspark.sql.streaming

Finished test(python3.9): pyspark.sql.readwriter (30s)

Finished test(python3.9): pyspark.sql.readwriter (28s)

Starting test(python3.9): pyspark.sql.types

Finished test(python3.9): pyspark.sql.types (10s)

Finished test(python3.9): pyspark.sql.types (9s)

Starting test(python3.9): pyspark.sql.udf

Finished test(python3.9): pyspark.sql.streaming (19s)

Finished test(python3.9): pyspark.sql.streaming (16s)

Starting test(python3.9): pyspark.sql.window

Finished test(python3.9): pyspark.sql.udf (17s)

Finished test(python3.9): pyspark.sql.window (16s)

Finished test(python3.9): pyspark.sql.window (17s)

Tests passed in 2979 seconds

Tests passed in 2827 seconds

Skipped tests in pyspark.sql.tests.test_arrow with pypy3:

test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_empty_partition (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_float_index (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_map_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_createDateFrame_with_category_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_filtered_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_no_partition_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_no_partition_toPandas (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_null_conversion (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_null_conversion (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyAr

test_pandas_round_trip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

test_pandas_self_destruct (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be insta

Saved diffs

Original text

Open file

Before
https://github.com/Yikun/spark/runs/2959151757?check_suite_focus=true

========================================================================
Running Spark unit tests
========================================================================

========================================================================
Running PySpark tests
========================================================================
Running PySpark tests. Output is in /__w/spark/spark/python/unit-tests.log
Will test against the following Python executables: ['python3.6', 'python3.9', 'pypy3']
Will test the following Python modules: ['pyspark-sql', 'pyspark-resource', 'pyspark-mllib']
python3.6 python_implementation is CPython
python3.6 version is: Python 3.6.13
python3.9 python_implementation is CPython
python3.9 version is: Python 3.9.5
pypy3 python_implementation is PyPy
pypy3 version is: Python 3.7.10 (7.3.5+dfsg-1~ppa1~ubuntu20.04, May 23 2021, 14:57:07)
[PyPy 7.3.5 with GCC 9.3.0]
Starting test(pypy3): pyspark.sql.tests.test_arrow
Starting test(pypy3): pyspark.sql.tests.test_catalog
Finished test(pypy3): pyspark.sql.tests.test_arrow (1s) ... 69 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_column
Finished test(pypy3): pyspark.sql.tests.test_catalog (24s)
Starting test(pypy3): pyspark.sql.tests.test_conf
Finished test(pypy3): pyspark.sql.tests.test_column (28s)
Starting test(pypy3): pyspark.sql.tests.test_context
Finished test(pypy3): pyspark.sql.tests.test_conf (15s)
Starting test(pypy3): pyspark.sql.tests.test_dataframe
Finished test(pypy3): pyspark.sql.tests.test_context (26s)
Starting test(pypy3): pyspark.sql.tests.test_datasources
Finished test(pypy3): pyspark.sql.tests.test_datasources (37s)
Starting test(pypy3): pyspark.sql.tests.test_functions
Finished test(pypy3): pyspark.sql.tests.test_dataframe (63s) ... 4 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_group
Finished test(pypy3): pyspark.sql.tests.test_group (27s)
Starting test(pypy3): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(pypy3): pyspark.sql.tests.test_pandas_cogrouped_map (1s) ... 16 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_grouped_map
Finished test(pypy3): pyspark.sql.tests.test_pandas_grouped_map (2s) ... 21 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_map
Finished test(pypy3): pyspark.sql.tests.test_pandas_map (2s) ... 7 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf (2s) ... 6 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_grouped_agg (2s) ... 13 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_scalar (2s) ... 51 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_typehints (2s) ... 10 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_window
Finished test(pypy3): pyspark.sql.tests.test_functions (53s)
Starting test(pypy3): pyspark.sql.tests.test_readwriter
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_window (1s) ... 14 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_serde
Finished test(pypy3): pyspark.sql.tests.test_serde (34s)
Starting test(pypy3): pyspark.sql.tests.test_session
Finished test(pypy3): pyspark.sql.tests.test_readwriter (45s)
Starting test(pypy3): pyspark.sql.tests.test_streaming
Finished test(pypy3): pyspark.sql.tests.test_session (40s)
Starting test(pypy3): pyspark.sql.tests.test_types
Finished test(pypy3): pyspark.sql.tests.test_streaming (51s)
Starting test(pypy3): pyspark.sql.tests.test_udf
Finished test(pypy3): pyspark.sql.tests.test_types (65s)
Starting test(pypy3): pyspark.sql.tests.test_utils
Finished test(pypy3): pyspark.sql.tests.test_utils (22s)
Starting test(python3.6): pyspark.mllib.tests.test_algorithms
Finished test(pypy3): pyspark.sql.tests.test_udf (67s)
Starting test(python3.6): pyspark.mllib.tests.test_feature
Finished test(python3.6): pyspark.mllib.tests.test_feature (43s)
Starting test(python3.6): pyspark.mllib.tests.test_linalg
Finished test(python3.6): pyspark.mllib.tests.test_algorithms (91s)
Starting test(python3.6): pyspark.mllib.tests.test_stat
Finished test(python3.6): pyspark.mllib.tests.test_stat (31s)
Starting test(python3.6): pyspark.mllib.tests.test_streaming_algorithms
Finished test(python3.6): pyspark.mllib.tests.test_linalg (83s)
Starting test(python3.6): pyspark.mllib.tests.test_util
Finished test(python3.6): pyspark.mllib.tests.test_util (14s)
Starting test(python3.6): pyspark.sql.tests.test_arrow
Finished test(python3.6): pyspark.sql.tests.test_arrow (49s)
Starting test(python3.6): pyspark.sql.tests.test_catalog
Finished test(python3.6): pyspark.sql.tests.test_catalog (26s)
Starting test(python3.6): pyspark.sql.tests.test_column
Finished test(python3.6): pyspark.sql.tests.test_column (27s)
Starting test(python3.6): pyspark.sql.tests.test_conf
Finished test(python3.6): pyspark.sql.tests.test_conf (15s)
Starting test(python3.6): pyspark.sql.tests.test_context
Finished test(python3.6): pyspark.sql.tests.test_context (20s)
Starting test(python3.6): pyspark.sql.tests.test_dataframe
Finished test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (187s)
Starting test(python3.6): pyspark.sql.tests.test_datasources
Finished test(python3.6): pyspark.sql.tests.test_datasources (31s)
Starting test(python3.6): pyspark.sql.tests.test_functions
Finished test(python3.6): pyspark.sql.tests.test_dataframe (63s) ... 2 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_group
Finished test(python3.6): pyspark.sql.tests.test_group (24s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(python3.6): pyspark.sql.tests.test_functions (48s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_grouped_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (46s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (52s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf
Finished test(python3.6): pyspark.sql.tests.test_pandas_map (33s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf (31s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (70s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints (33s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_window
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (112s)
Starting test(python3.6): pyspark.sql.tests.test_readwriter
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (50s)
Starting test(python3.6): pyspark.sql.tests.test_serde
Finished test(python3.6): pyspark.sql.tests.test_readwriter (44s)
Starting test(python3.6): pyspark.sql.tests.test_session
Finished test(python3.6): pyspark.sql.tests.test_serde (33s)
Starting test(python3.6): pyspark.sql.tests.test_streaming
Finished test(python3.6): pyspark.sql.tests.test_session (38s)
Starting test(python3.6): pyspark.sql.tests.test_types
Finished test(python3.6): pyspark.sql.tests.test_streaming (45s)
Starting test(python3.6): pyspark.sql.tests.test_udf
Finished test(python3.6): pyspark.sql.tests.test_types (53s)
Starting test(python3.6): pyspark.sql.tests.test_utils
Finished test(python3.6): pyspark.sql.tests.test_utils (17s)
Starting test(python3.9): pyspark.mllib.tests.test_algorithms
Finished test(python3.6): pyspark.sql.tests.test_udf (58s)
Starting test(python3.9): pyspark.mllib.tests.test_feature
Finished test(python3.9): pyspark.mllib.tests.test_feature (41s)
Starting test(python3.9): pyspark.mllib.tests.test_linalg
Finished test(python3.9): pyspark.mllib.tests.test_algorithms (91s)
Starting test(python3.9): pyspark.mllib.tests.test_stat
Finished test(python3.9): pyspark.mllib.tests.test_stat (30s)
Starting test(python3.9): pyspark.mllib.tests.test_streaming_algorithms
Finished test(python3.9): pyspark.mllib.tests.test_linalg (81s)
Starting test(python3.9): pyspark.mllib.tests.test_util
Finished test(python3.9): pyspark.mllib.tests.test_util (15s)
Starting test(python3.9): pyspark.sql.tests.test_arrow
Finished test(python3.9): pyspark.sql.tests.test_arrow (46s)
Starting test(python3.9): pyspark.sql.tests.test_catalog
Finished test(python3.9): pyspark.sql.tests.test_catalog (25s)
Starting test(python3.9): pyspark.sql.tests.test_column
Finished test(python3.9): pyspark.sql.tests.test_column (27s)
Starting test(python3.9): pyspark.sql.tests.test_conf
Finished test(python3.9): pyspark.sql.tests.test_conf (14s)
Starting test(python3.9): pyspark.sql.tests.test_context
Finished test(python3.9): pyspark.sql.tests.test_context (20s)
Starting test(python3.9): pyspark.sql.tests.test_dataframe
Finished test(python3.9): pyspark.mllib.tests.test_streaming_algorithms (181s)
Starting test(python3.9): pyspark.sql.tests.test_datasources
Finished test(python3.9): pyspark.sql.tests.test_datasources (28s)
Starting test(python3.9): pyspark.sql.tests.test_functions
Finished test(python3.9): pyspark.sql.tests.test_dataframe (59s) ... 2 tests were skipped
Starting test(python3.9): pyspark.sql.tests.test_group
Finished test(python3.9): pyspark.sql.tests.test_group (21s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(python3.9): pyspark.sql.tests.test_functions (48s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_grouped_map
Finished test(python3.9): pyspark.sql.tests.test_pandas_cogrouped_map (46s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_map
Finished test(python3.9): pyspark.sql.tests.test_pandas_grouped_map (52s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf
Finished test(python3.9): pyspark.sql.tests.test_pandas_map (33s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf (31s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_grouped_agg (67s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_typehints (36s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_window
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_scalar (107s)
Starting test(python3.9): pyspark.sql.tests.test_readwriter
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_window (52s)
Starting test(python3.9): pyspark.sql.tests.test_serde
Finished test(python3.9): pyspark.sql.tests.test_readwriter (43s)
Starting test(python3.9): pyspark.sql.tests.test_session
Finished test(python3.9): pyspark.sql.tests.test_serde (33s)
Starting test(python3.9): pyspark.sql.tests.test_streaming
Finished test(python3.9): pyspark.sql.tests.test_session (37s)
Starting test(python3.9): pyspark.sql.tests.test_types
Finished test(python3.9): pyspark.sql.tests.test_streaming (45s)
Starting test(python3.9): pyspark.sql.tests.test_udf
Finished test(python3.9): pyspark.sql.tests.test_types (53s)
Starting test(python3.9): pyspark.sql.tests.test_utils
Finished test(python3.9): pyspark.sql.tests.test_utils (16s)
Starting test(pypy3): pyspark.resource.tests.test_resources
Finished test(python3.9): pyspark.sql.tests.test_udf (56s)
Starting test(pypy3): pyspark.sql.avro.functions
Finished test(pypy3): pyspark.resource.tests.test_resources (8s)
Starting test(pypy3): pyspark.sql.catalog
Finished test(pypy3): pyspark.sql.avro.functions (19s)
Starting test(pypy3): pyspark.sql.column
Finished test(pypy3): pyspark.sql.catalog (19s)
Starting test(pypy3): pyspark.sql.conf
Finished test(pypy3): pyspark.sql.conf (8s)
Starting test(pypy3): pyspark.sql.context
Finished test(pypy3): pyspark.sql.column (36s)
Starting test(pypy3): pyspark.sql.dataframe
Finished test(pypy3): pyspark.sql.context (32s)
Starting test(pypy3): pyspark.sql.functions
Finished test(pypy3): pyspark.sql.dataframe (71s)
Starting test(pypy3): pyspark.sql.group
Finished test(pypy3): pyspark.sql.functions (88s)
Starting test(pypy3): pyspark.sql.pandas.conversion
Finished test(pypy3): pyspark.sql.group (36s)
Starting test(pypy3): pyspark.sql.pandas.group_ops
Finished test(pypy3): pyspark.sql.pandas.conversion (10s)
Starting test(pypy3): pyspark.sql.pandas.map_ops
Finished test(pypy3): pyspark.sql.pandas.group_ops (13s)
Starting test(pypy3): pyspark.sql.pandas.serializers
Finished test(pypy3): pyspark.sql.pandas.serializers (0s)
Starting test(pypy3): pyspark.sql.pandas.typehints
Finished test(pypy3): pyspark.sql.pandas.typehints (0s)
Starting test(pypy3): pyspark.sql.pandas.types
Finished test(pypy3): pyspark.sql.pandas.types (0s)
Starting test(pypy3): pyspark.sql.pandas.utils
Finished test(pypy3): pyspark.sql.pandas.map_ops (12s)
Starting test(pypy3): pyspark.sql.readwriter
Finished test(pypy3): pyspark.sql.pandas.utils (0s)
Starting test(pypy3): pyspark.sql.session
Finished test(pypy3): pyspark.sql.session (27s)
Starting test(pypy3): pyspark.sql.streaming
Finished test(pypy3): pyspark.sql.readwriter (31s)
Starting test(pypy3): pyspark.sql.types
Finished test(pypy3): pyspark.sql.types (10s)
Starting test(pypy3): pyspark.sql.udf
Finished test(pypy3): pyspark.sql.streaming (19s)
Starting test(pypy3): pyspark.sql.window
Finished test(pypy3): pyspark.sql.udf (18s)
Starting test(python3.6): pyspark.mllib.classification
Finished test(pypy3): pyspark.sql.window (20s)
Starting test(python3.6): pyspark.mllib.clustering
Finished test(python3.6): pyspark.mllib.classification (56s)
Starting test(python3.6): pyspark.mllib.evaluation
Finished test(python3.6): pyspark.mllib.clustering (73s)
Starting test(python3.6): pyspark.mllib.feature
Finished test(python3.6): pyspark.mllib.evaluation (34s)
Starting test(python3.6): pyspark.mllib.fpm
Finished test(python3.6): pyspark.mllib.fpm (28s)
Starting test(python3.6): pyspark.mllib.linalg.__init__
Finished test(python3.6): pyspark.mllib.linalg.__init__ (0s)
Starting test(python3.6): pyspark.mllib.linalg.distributed
Finished test(python3.6): pyspark.mllib.feature (38s)
Starting test(python3.6): pyspark.mllib.random
Finished test(python3.6): pyspark.mllib.random (15s)
Starting test(python3.6): pyspark.mllib.recommendation
Finished test(python3.6): pyspark.mllib.recommendation (61s)
Starting test(python3.6): pyspark.mllib.regression
Finished test(python3.6): pyspark.mllib.linalg.distributed (79s)
Starting test(python3.6): pyspark.mllib.stat.KernelDensity
Finished test(python3.6): pyspark.mllib.stat.KernelDensity (0s)
Starting test(python3.6): pyspark.mllib.stat._statistics
Finished test(python3.6): pyspark.mllib.stat._statistics (26s)
Starting test(python3.6): pyspark.mllib.tree
Finished test(python3.6): pyspark.mllib.regression (41s)
Starting test(python3.6): pyspark.mllib.util
Finished test(python3.6): pyspark.mllib.tree (27s)
Starting test(python3.6): pyspark.resource.tests.test_resources
Finished test(python3.6): pyspark.resource.tests.test_resources (8s)
Starting test(python3.6): pyspark.sql.avro.functions
Finished test(python3.6): pyspark.mllib.util (28s)
Starting test(python3.6): pyspark.sql.catalog
Finished test(python3.6): pyspark.sql.avro.functions (19s)
Starting test(python3.6): pyspark.sql.column
Finished test(python3.6): pyspark.sql.catalog (17s)
Starting test(python3.6): pyspark.sql.conf
Finished test(python3.6): pyspark.sql.conf (8s)
Starting test(python3.6): pyspark.sql.context
Finished test(python3.6): pyspark.sql.column (32s)
Starting test(python3.6): pyspark.sql.dataframe
Finished test(python3.6): pyspark.sql.context (28s)
Starting test(python3.6): pyspark.sql.functions
Finished test(python3.6): pyspark.sql.dataframe (64s)
Starting test(python3.6): pyspark.sql.group
Finished test(python3.6): pyspark.sql.functions (73s)
Starting test(python3.6): pyspark.sql.pandas.conversion
Finished test(python3.6): pyspark.sql.pandas.conversion (10s)
Starting test(python3.6): pyspark.sql.pandas.group_ops
Finished test(python3.6): pyspark.sql.group (28s)
Starting test(python3.6): pyspark.sql.pandas.map_ops
Finished test(python3.6): pyspark.sql.pandas.group_ops (12s)
Starting test(python3.6): pyspark.sql.pandas.serializers
Finished test(python3.6): pyspark.sql.pandas.serializers (0s)
Starting test(python3.6): pyspark.sql.pandas.typehints
Finished test(python3.6): pyspark.sql.pandas.typehints (0s)
Starting test(python3.6): pyspark.sql.pandas.types
Finished test(python3.6): pyspark.sql.pandas.types (0s)
Starting test(python3.6): pyspark.sql.pandas.utils
Finished test(python3.6): pyspark.sql.pandas.utils (0s)
Starting test(python3.6): pyspark.sql.readwriter
Finished test(python3.6): pyspark.sql.pandas.map_ops (12s)
Starting test(python3.6): pyspark.sql.session
Finished test(python3.6): pyspark.sql.session (25s)
Starting test(python3.6): pyspark.sql.streaming
Finished test(python3.6): pyspark.sql.readwriter (31s)
Starting test(python3.6): pyspark.sql.types
Finished test(python3.6): pyspark.sql.types (10s)
Starting test(python3.6): pyspark.sql.udf
Finished test(python3.6): pyspark.sql.streaming (19s)
Starting test(python3.6): pyspark.sql.window
Finished test(python3.6): pyspark.sql.udf (19s)
Starting test(python3.9): pyspark.mllib.classification
Finished test(python3.6): pyspark.sql.window (20s)
Starting test(python3.9): pyspark.mllib.clustering
Finished test(python3.9): pyspark.mllib.classification (59s)
Starting test(python3.9): pyspark.mllib.evaluation
Finished test(python3.9): pyspark.mllib.clustering (72s)
Starting test(python3.9): pyspark.mllib.feature
Finished test(python3.9): pyspark.mllib.evaluation (30s)
Starting test(python3.9): pyspark.mllib.fpm
Finished test(python3.9): pyspark.mllib.feature (38s)
Starting test(python3.9): pyspark.mllib.linalg.__init__
Finished test(python3.9): pyspark.mllib.linalg.__init__ (0s)
Starting test(python3.9): pyspark.mllib.linalg.distributed
Finished test(python3.9): pyspark.mllib.fpm (30s)
Starting test(python3.9): pyspark.mllib.random
Finished test(python3.9): pyspark.mllib.random (15s)
Starting test(python3.9): pyspark.mllib.recommendation
Finished test(python3.9): pyspark.mllib.linalg.distributed (78s)
Starting test(python3.9): pyspark.mllib.regression
Finished test(python3.9): pyspark.mllib.recommendation (59s)
Starting test(python3.9): pyspark.mllib.stat.KernelDensity
Finished test(python3.9): pyspark.mllib.stat.KernelDensity (0s)
Starting test(python3.9): pyspark.mllib.stat._statistics
Finished test(python3.9): pyspark.mllib.stat._statistics (25s)
Starting test(python3.9): pyspark.mllib.tree
Finished test(python3.9): pyspark.mllib.regression (41s)
Starting test(python3.9): pyspark.mllib.util
Finished test(python3.9): pyspark.mllib.tree (28s)
Starting test(python3.9): pyspark.resource.tests.test_resources
Finished test(python3.9): pyspark.resource.tests.test_resources (8s)
Starting test(python3.9): pyspark.sql.avro.functions
Finished test(python3.9): pyspark.mllib.util (27s)
Starting test(python3.9): pyspark.sql.catalog
Finished test(python3.9): pyspark.sql.avro.functions (19s)
Starting test(python3.9): pyspark.sql.column
Finished test(python3.9): pyspark.sql.catalog (17s)
Starting test(python3.9): pyspark.sql.conf
Finished test(python3.9): pyspark.sql.conf (8s)
Starting test(python3.9): pyspark.sql.context
Finished test(python3.9): pyspark.sql.column (33s)
Starting test(python3.9): pyspark.sql.dataframe
Finished test(python3.9): pyspark.sql.context (28s)
Starting test(python3.9): pyspark.sql.functions
Finished test(python3.9): pyspark.sql.dataframe (63s)
Starting test(python3.9): pyspark.sql.group
Finished test(python3.9): pyspark.sql.functions (79s)
Starting test(python3.9): pyspark.sql.pandas.conversion
Finished test(python3.9): pyspark.sql.group (30s)
Starting test(python3.9): pyspark.sql.pandas.group_ops
Finished test(python3.9): pyspark.sql.pandas.conversion (9s)
Starting test(python3.9): pyspark.sql.pandas.map_ops
Finished test(python3.9): pyspark.sql.pandas.group_ops (12s)
Starting test(python3.9): pyspark.sql.pandas.serializers
Finished test(python3.9): pyspark.sql.pandas.serializers (0s)
Starting test(python3.9): pyspark.sql.pandas.typehints
Finished test(python3.9): pyspark.sql.pandas.typehints (0s)
Starting test(python3.9): pyspark.sql.pandas.types
Finished test(python3.9): pyspark.sql.pandas.map_ops (11s)
Starting test(python3.9): pyspark.sql.pandas.utils
Finished test(python3.9): pyspark.sql.pandas.types (0s)
Starting test(python3.9): pyspark.sql.readwriter
Finished test(python3.9): pyspark.sql.pandas.utils (0s)
Starting test(python3.9): pyspark.sql.session
Finished test(python3.9): pyspark.sql.session (25s)
Starting test(python3.9): pyspark.sql.streaming
Finished test(python3.9): pyspark.sql.readwriter (30s)
Starting test(python3.9): pyspark.sql.types
Finished test(python3.9): pyspark.sql.types (10s)
Starting test(python3.9): pyspark.sql.udf
Finished test(python3.9): pyspark.sql.streaming (19s)
Starting test(python3.9): pyspark.sql.window
Finished test(python3.9): pyspark.sql.udf (17s)
Finished test(python3.9): pyspark.sql.window (16s)
Tests passed in 2979 seconds

Skipped tests in pyspark.sql.tests.test_arrow with pypy3:
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_empty_partition (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_float_index (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_map_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDateFrame_with_category_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_filtered_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_no_partition_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_no_partition_toPandas (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_null_conversion (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_round_trip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_self_destruct (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_propagates_spark_exception (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_timestamp_nat (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_batch_order (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_with_map_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_with_map_type_nulls (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_empty_partition (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_float_index (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_map_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_createDateFrame_with_category_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_filtered_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_no_partition_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_no_partition_toPandas (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_null_conversion (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_round_trip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_self_destruct (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_propagates_spark_exception (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_timestamp_nat (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_batch_order (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_with_map_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_toPandas_with_map_type_nulls (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_exception_by_max_results (pyspark.sql.tests.test_arrow.MaxResultArrowTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_dataframe with pypy3:
    test_create_dataframe_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped 'Required Pandas was found.'
    test_to_pandas_on_spark (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_to_pandas_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped 'Required Pandas was found.'
    test_query_execution_listener_on_collect_with_arrow (pyspark.sql.tests.test_dataframe.QueryExecutionListenerTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_cogrouped_map with pypy3:
    test_case_insensitive_grouping_column (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_complex_group_by (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_different_schemas (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_empty_group_by (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_left_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_mixed_scalar_udfs_followed_by_cogrouby_apply (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_right_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_self_join (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_simple (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_with_key_complex (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_with_key_left (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_with_key_left_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_with_key_right (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_with_key_right_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_wrong_args (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_wrong_return_type (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_grouped_map with pypy3:
    test_array_type_correct (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_case_insensitive_grouping_column (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_coerce (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_column_order (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_complex_groupby (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_datatype_string (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_decorator (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_empty_groupby (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_grouped_over_window (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_grouped_over_window_with_key (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_mixed_scalar_udfs_followed_by_groupby_apply (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_positional_assignment_conf (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_register_grouped_map_udf (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_self_join_with_pandas (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_supported_types (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_udf_with_key (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_unsupported_types (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_wrong_args (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_wrong_return_type (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_map with pypy3:
    test_chain_map_partitions_in_pandas (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_different_output_length (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_empty_iterator (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_empty_rows (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_map_partitions_in_pandas (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_multiple_columns (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_self_join (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf with pypy3:
    test_pandas_udf_arrow_overflow (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_udf_basic (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_udf_decorator (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_udf_detect_unsafe_type_conversion (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_stopiteration_in_udf (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_udf_wrong_arg (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_grouped_agg with pypy3:
    test_alias (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_array_type (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_complex_expressions (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_complex_groupby (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_grouped_without_group_by_clause (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_invalid_args (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_manual (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_no_predicate_pushdown_through (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_retain_group_columns (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_unsupported_types (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_scalar with pypy3:
    test_datasource_with_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_mixed_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_mixed_udf_and_sql (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_nondeterministic_vectorized_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_nondeterministic_vectorized_udf_in_aggregate (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_udf_nested_arrays (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_pandas_udf_tokenize (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_register_nondeterministic_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_scalar_iter_udf_close (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_scalar_iter_udf_close_early (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_scalar_iter_udf_init (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_type_annotation (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_udf_category_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_array_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_chained (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_chained_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_check_config (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_datatype_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_dates (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_decorator (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_exception (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_invalid_length (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_map_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_nested_struct (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_array (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_binary (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_boolean (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_byte (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_decimal (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_double (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_float (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_int (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_long (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_short (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_null_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_return_scalar (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_return_timestamp_tz (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_string_in_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_struct_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_struct_with_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_timestamps (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_timestamps_respect_session_timezone (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_unsupported_types (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_varargs (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_vectorized_udf_wrong_return_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_typehints with pypy3:
    test_group_agg_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_ignore_type_hint_in_cogroup_apply_in_pandas (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_ignore_type_hint_in_group_apply_in_pandas (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_ignore_type_hint_in_map_in_pandas (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_scalar_iter_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_scalar_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_type_annotation_group_agg (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_type_annotation_negative (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_type_annotation_scalar (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_type_annotation_scalar_iter (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_window with pypy3:
    test_array_type (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_bounded_mixed (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_bounded_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_growing_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_invalid_args (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_mixed_sql (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_mixed_sql_and_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_mixed_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_multiple_udfs (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_replace_existing (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_shrinking_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_sliding_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'
    test_without_partitionBy (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 1.0.0 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_dataframe with python3.6:
      test_create_dataframe_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... SKIP (0.000s)
      test_to_pandas_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... SKIP (0.000s)

Skipped tests in pyspark.sql.tests.test_dataframe with python3.9:
      test_create_dataframe_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... SKIP (0.000s)
      test_to_pandas_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... SKIP (0.000s)

========================================================================
Running PySpark packaging tests
========================================================================
Constructing virtual env for testing
Using conda virtual environments
Testing pip installation with python 3.6
Using /tmp/tmp.it3T0i4XvZ for virtualenv
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

Changed text

Open file

After
https://github.com/Yikun/spark/runs/2968285996

[success] Total time: 206 s (03:26), completed Jul 2, 2021 1:54:02 AM

========================================================================
Running Spark unit tests
========================================================================

========================================================================
Running PySpark tests
========================================================================
'PYARROW_IGNORE_TIMEZONE' environment variable was not set. It is required to set this environment variable to '1' in both driver and executor sides if you use pyarrow>=2.0.0. pandas-on-Spark will set it for you but it does not work if there is a Spark context already launched.
Running PySpark tests. Output is in /__w/spark/spark/python/unit-tests.log
Will test against the following Python executables: ['python3.6', 'python3.9', 'pypy3']
Will test the following Python modules: ['pyspark-sql', 'pyspark-resource', 'pyspark-mllib']
python3.6 python_implementation is CPython
python3.6 version is: Python 3.6.13
python3.9 python_implementation is CPython
python3.9 version is: Python 3.9.5
pypy3 python_implementation is PyPy
pypy3 version is: Python 3.7.10 (7.3.5+dfsg-1~ppa1~ubuntu20.04, May 23 2021, 14:57:07)
[PyPy 7.3.5 with GCC 9.3.0]
Starting test(pypy3): pyspark.sql.tests.test_catalog
Starting test(pypy3): pyspark.sql.tests.test_arrow
Finished test(pypy3): pyspark.sql.tests.test_arrow (1s) ... 69 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_column
Finished test(pypy3): pyspark.sql.tests.test_catalog (21s)
Starting test(pypy3): pyspark.sql.tests.test_conf
Finished test(pypy3): pyspark.sql.tests.test_column (27s)
Starting test(pypy3): pyspark.sql.tests.test_context
Finished test(pypy3): pyspark.sql.tests.test_conf (14s)
Starting test(pypy3): pyspark.sql.tests.test_dataframe
Finished test(pypy3): pyspark.sql.tests.test_context (26s)
Starting test(pypy3): pyspark.sql.tests.test_datasources
Finished test(pypy3): pyspark.sql.tests.test_datasources (34s)
Starting test(pypy3): pyspark.sql.tests.test_functions
Finished test(pypy3): pyspark.sql.tests.test_dataframe (58s) ... 4 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_group
Finished test(pypy3): pyspark.sql.tests.test_group (25s)
Starting test(pypy3): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(pypy3): pyspark.sql.tests.test_pandas_cogrouped_map (2s) ... 16 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_grouped_map
Finished test(pypy3): pyspark.sql.tests.test_pandas_grouped_map (1s) ... 21 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_map
Finished test(pypy3): pyspark.sql.tests.test_pandas_map (2s) ... 7 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf (2s) ... 6 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_grouped_agg (2s) ... 13 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_scalar (2s) ... 51 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_typehints (2s) ... 10 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_window
Finished test(pypy3): pyspark.sql.tests.test_pandas_udf_window (2s) ... 14 tests were skipped
Starting test(pypy3): pyspark.sql.tests.test_readwriter
Finished test(pypy3): pyspark.sql.tests.test_functions (49s)
Starting test(pypy3): pyspark.sql.tests.test_serde
Finished test(pypy3): pyspark.sql.tests.test_serde (32s)
Starting test(pypy3): pyspark.sql.tests.test_session
Finished test(pypy3): pyspark.sql.tests.test_readwriter (38s)
Starting test(pypy3): pyspark.sql.tests.test_streaming
Finished test(pypy3): pyspark.sql.tests.test_session (36s)
Starting test(pypy3): pyspark.sql.tests.test_types
Finished test(pypy3): pyspark.sql.tests.test_streaming (46s)
Starting test(pypy3): pyspark.sql.tests.test_udf
Finished test(pypy3): pyspark.sql.tests.test_types (71s)
Starting test(pypy3): pyspark.sql.tests.test_utils
Finished test(pypy3): pyspark.sql.tests.test_udf (71s)
Starting test(python3.6): pyspark.mllib.tests.test_algorithms
Finished test(pypy3): pyspark.sql.tests.test_utils (18s)
Starting test(python3.6): pyspark.mllib.tests.test_feature
Finished test(python3.6): pyspark.mllib.tests.test_feature (38s)
Starting test(python3.6): pyspark.mllib.tests.test_linalg
Finished test(python3.6): pyspark.mllib.tests.test_algorithms (85s)
Starting test(python3.6): pyspark.mllib.tests.test_stat
Finished test(python3.6): pyspark.mllib.tests.test_stat (29s)
Starting test(python3.6): pyspark.mllib.tests.test_streaming_algorithms
Finished test(python3.6): pyspark.mllib.tests.test_linalg (77s)
Starting test(python3.6): pyspark.mllib.tests.test_util
Finished test(python3.6): pyspark.mllib.tests.test_util (14s)
Starting test(python3.6): pyspark.sql.tests.test_arrow
Finished test(python3.6): pyspark.sql.tests.test_arrow (51s)
Starting test(python3.6): pyspark.sql.tests.test_catalog
Finished test(python3.6): pyspark.sql.tests.test_catalog (27s)
Starting test(python3.6): pyspark.sql.tests.test_column
Finished test(python3.6): pyspark.sql.tests.test_column (27s)
Starting test(python3.6): pyspark.sql.tests.test_conf
Finished test(python3.6): pyspark.sql.tests.test_conf (13s)
Starting test(python3.6): pyspark.sql.tests.test_context
Finished test(python3.6): pyspark.sql.tests.test_context (19s)
Starting test(python3.6): pyspark.sql.tests.test_dataframe
Finished test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (190s)
Starting test(python3.6): pyspark.sql.tests.test_datasources
Finished test(python3.6): pyspark.sql.tests.test_dataframe (59s) ... 2 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_functions
Finished test(python3.6): pyspark.sql.tests.test_datasources (28s)
Starting test(python3.6): pyspark.sql.tests.test_group
Finished test(python3.6): pyspark.sql.tests.test_group (21s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(python3.6): pyspark.sql.tests.test_functions (45s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_grouped_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (43s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_map (34s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf
Finished test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (55s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf (31s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (70s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints (33s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_window
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (105s)
Starting test(python3.6): pyspark.sql.tests.test_readwriter
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (48s)
Starting test(python3.6): pyspark.sql.tests.test_serde
Finished test(python3.6): pyspark.sql.tests.test_readwriter (40s)
Starting test(python3.6): pyspark.sql.tests.test_session
Finished test(python3.6): pyspark.sql.tests.test_serde (33s)
Starting test(python3.6): pyspark.sql.tests.test_streaming
Finished test(python3.6): pyspark.sql.tests.test_session (36s)
Starting test(python3.6): pyspark.sql.tests.test_types
Finished test(python3.6): pyspark.sql.tests.test_streaming (45s)
Starting test(python3.6): pyspark.sql.tests.test_udf
Finished test(python3.6): pyspark.sql.tests.test_types (52s)
Starting test(python3.6): pyspark.sql.tests.test_utils
Finished test(python3.6): pyspark.sql.tests.test_utils (15s)
Starting test(python3.9): pyspark.mllib.tests.test_algorithms
Finished test(python3.6): pyspark.sql.tests.test_udf (56s)
Starting test(python3.9): pyspark.mllib.tests.test_feature
Finished test(python3.9): pyspark.mllib.tests.test_feature (40s)
Starting test(python3.9): pyspark.mllib.tests.test_linalg
Finished test(python3.9): pyspark.mllib.tests.test_algorithms (82s)
Starting test(python3.9): pyspark.mllib.tests.test_stat
Finished test(python3.9): pyspark.mllib.tests.test_stat (31s)
Starting test(python3.9): pyspark.mllib.tests.test_streaming_algorithms
Finished test(python3.9): pyspark.mllib.tests.test_linalg (77s)
Starting test(python3.9): pyspark.mllib.tests.test_util
Finished test(python3.9): pyspark.mllib.tests.test_util (11s)
Starting test(python3.9): pyspark.sql.tests.test_arrow
Finished test(python3.9): pyspark.sql.tests.test_arrow (47s)
Starting test(python3.9): pyspark.sql.tests.test_catalog
Finished test(python3.9): pyspark.sql.tests.test_catalog (20s)
Starting test(python3.9): pyspark.sql.tests.test_column
Finished test(python3.9): pyspark.sql.tests.test_column (23s)
Starting test(python3.9): pyspark.sql.tests.test_conf
Finished test(python3.9): pyspark.sql.tests.test_conf (12s)
Starting test(python3.9): pyspark.sql.tests.test_context
Finished test(python3.9): pyspark.sql.tests.test_context (18s)
Starting test(python3.9): pyspark.sql.tests.test_dataframe
Finished test(python3.9): pyspark.mllib.tests.test_streaming_algorithms (172s)
Starting test(python3.9): pyspark.sql.tests.test_datasources
Finished test(python3.9): pyspark.sql.tests.test_datasources (29s)
Starting test(python3.9): pyspark.sql.tests.test_functions
Finished test(python3.9): pyspark.sql.tests.test_dataframe (58s) ... 2 tests were skipped
Starting test(python3.9): pyspark.sql.tests.test_group
Finished test(python3.9): pyspark.sql.tests.test_group (21s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(python3.9): pyspark.sql.tests.test_functions (45s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_grouped_map
Finished test(python3.9): pyspark.sql.tests.test_pandas_cogrouped_map (40s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_map
Finished test(python3.9): pyspark.sql.tests.test_pandas_map (29s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf
Finished test(python3.9): pyspark.sql.tests.test_pandas_grouped_map (49s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf (25s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_grouped_agg (63s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_typehints (29s)
Starting test(python3.9): pyspark.sql.tests.test_pandas_udf_window
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_scalar (98s)
Starting test(python3.9): pyspark.sql.tests.test_readwriter
Finished test(python3.9): pyspark.sql.tests.test_pandas_udf_window (46s)
Starting test(python3.9): pyspark.sql.tests.test_serde
Finished test(python3.9): pyspark.sql.tests.test_readwriter (44s)
Starting test(python3.9): pyspark.sql.tests.test_session
Finished test(python3.9): pyspark.sql.tests.test_serde (34s)
Starting test(python3.9): pyspark.sql.tests.test_streaming
Finished test(python3.9): pyspark.sql.tests.test_session (37s)
Starting test(python3.9): pyspark.sql.tests.test_types
Finished test(python3.9): pyspark.sql.tests.test_streaming (46s)
Starting test(python3.9): pyspark.sql.tests.test_udf
Finished test(python3.9): pyspark.sql.tests.test_types (53s)
Starting test(python3.9): pyspark.sql.tests.test_utils
Finished test(python3.9): pyspark.sql.tests.test_utils (16s)
Starting test(pypy3): pyspark.resource.tests.test_resources
Finished test(python3.9): pyspark.sql.tests.test_udf (58s)
Starting test(pypy3): pyspark.sql.avro.functions
Finished test(pypy3): pyspark.resource.tests.test_resources (7s)
Starting test(pypy3): pyspark.sql.catalog
Finished test(pypy3): pyspark.sql.avro.functions (20s)
Starting test(pypy3): pyspark.sql.column
Finished test(pypy3): pyspark.sql.catalog (20s)
Starting test(pypy3): pyspark.sql.conf
Finished test(pypy3): pyspark.sql.conf (8s)
Starting test(pypy3): pyspark.sql.context
Finished test(pypy3): pyspark.sql.column (36s)
Starting test(pypy3): pyspark.sql.dataframe
Finished test(pypy3): pyspark.sql.context (32s)
Starting test(pypy3): pyspark.sql.functions
Finished test(pypy3): pyspark.sql.dataframe (73s)
Starting test(pypy3): pyspark.sql.group
Finished test(pypy3): pyspark.sql.functions (88s)
Starting test(pypy3): pyspark.sql.pandas.conversion
Finished test(pypy3): pyspark.sql.group (33s)
Starting test(pypy3): pyspark.sql.pandas.group_ops
Finished test(pypy3): pyspark.sql.pandas.conversion (11s)
Starting test(pypy3): pyspark.sql.pandas.map_ops
Finished test(pypy3): pyspark.sql.pandas.group_ops (12s)
Starting test(pypy3): pyspark.sql.pandas.serializers
Finished test(pypy3): pyspark.sql.pandas.serializers (0s)
Starting test(pypy3): pyspark.sql.pandas.typehints
Finished test(pypy3): pyspark.sql.pandas.map_ops (12s)
Starting test(pypy3): pyspark.sql.pandas.types
Finished test(pypy3): pyspark.sql.pandas.typehints (0s)
Starting test(pypy3): pyspark.sql.pandas.utils
Finished test(pypy3): pyspark.sql.pandas.types (0s)
Starting test(pypy3): pyspark.sql.readwriter
Finished test(pypy3): pyspark.sql.pandas.utils (0s)
Starting test(pypy3): pyspark.sql.session
Finished test(pypy3): pyspark.sql.session (26s)
Starting test(pypy3): pyspark.sql.streaming
Finished test(pypy3): pyspark.sql.readwriter (29s)
Starting test(pypy3): pyspark.sql.types
Finished test(pypy3): pyspark.sql.types (9s)
Starting test(pypy3): pyspark.sql.udf
Finished test(pypy3): pyspark.sql.streaming (15s)
Starting test(pypy3): pyspark.sql.window
Finished test(pypy3): pyspark.sql.udf (17s)
Starting test(python3.6): pyspark.mllib.classification
Finished test(pypy3): pyspark.sql.window (18s)
Starting test(python3.6): pyspark.mllib.clustering
Finished test(python3.6): pyspark.mllib.classification (56s)
Starting test(python3.6): pyspark.mllib.evaluation
Finished test(python3.6): pyspark.mllib.clustering (67s)
Starting test(python3.6): pyspark.mllib.feature
Finished test(python3.6): pyspark.mllib.evaluation (26s)
Starting test(python3.6): pyspark.mllib.fpm
Finished test(python3.6): pyspark.mllib.feature (32s)
Starting test(python3.6): pyspark.mllib.linalg.__init__
Finished test(python3.6): pyspark.mllib.linalg.__init__ (0s)
Starting test(python3.6): pyspark.mllib.linalg.distributed
Finished test(python3.6): pyspark.mllib.fpm (24s)
Starting test(python3.6): pyspark.mllib.random
Finished test(python3.6): pyspark.mllib.random (13s)
Starting test(python3.6): pyspark.mllib.recommendation
Finished test(python3.6): pyspark.mllib.linalg.distributed (71s)
Starting test(python3.6): pyspark.mllib.regression
Finished test(python3.6): pyspark.mllib.recommendation (56s)
Starting test(python3.6): pyspark.mllib.stat.KernelDensity
Finished test(python3.6): pyspark.mllib.stat.KernelDensity (0s)
Starting test(python3.6): pyspark.mllib.stat._statistics
Finished test(python3.6): pyspark.mllib.stat._statistics (22s)
Starting test(python3.6): pyspark.mllib.tree
Finished test(python3.6): pyspark.mllib.regression (35s)
Starting test(python3.6): pyspark.mllib.util
Finished test(python3.6): pyspark.mllib.tree (24s)
Starting test(python3.6): pyspark.resource.tests.test_resources
Finished test(python3.6): pyspark.resource.tests.test_resources (7s)
Starting test(python3.6): pyspark.sql.avro.functions
Finished test(python3.6): pyspark.mllib.util (25s)
Starting test(python3.6): pyspark.sql.catalog
Finished test(python3.6): pyspark.sql.avro.functions (19s)
Starting test(python3.6): pyspark.sql.column
Finished test(python3.6): pyspark.sql.catalog (16s)
Starting test(python3.6): pyspark.sql.conf
Finished test(python3.6): pyspark.sql.conf (8s)
Starting test(python3.6): pyspark.sql.context
Finished test(python3.6): pyspark.sql.column (33s)
Starting test(python3.6): pyspark.sql.dataframe
Finished test(python3.6): pyspark.sql.context (26s)
Starting test(python3.6): pyspark.sql.functions
Finished test(python3.6): pyspark.sql.dataframe (57s)
Starting test(python3.6): pyspark.sql.group
Finished test(python3.6): pyspark.sql.functions (69s)
Starting test(python3.6): pyspark.sql.pandas.conversion
Finished test(python3.6): pyspark.sql.group (27s)
Starting test(python3.6): pyspark.sql.pandas.group_ops
Finished test(python3.6): pyspark.sql.pandas.conversion (9s)
Starting test(python3.6): pyspark.sql.pandas.map_ops
Finished test(python3.6): pyspark.sql.pandas.group_ops (11s)
Starting test(python3.6): pyspark.sql.pandas.serializers
Finished test(python3.6): pyspark.sql.pandas.map_ops (11s)
Starting test(python3.6): pyspark.sql.pandas.typehints
Finished test(python3.6): pyspark.sql.pandas.serializers (0s)
Starting test(python3.6): pyspark.sql.pandas.types
Finished test(python3.6): pyspark.sql.pandas.typehints (0s)
Starting test(python3.6): pyspark.sql.pandas.utils
Finished test(python3.6): pyspark.sql.pandas.types (0s)
Starting test(python3.6): pyspark.sql.readwriter
Finished test(python3.6): pyspark.sql.pandas.utils (0s)
Starting test(python3.6): pyspark.sql.session
Finished test(python3.6): pyspark.sql.session (23s)
Starting test(python3.6): pyspark.sql.streaming
Finished test(python3.6): pyspark.sql.readwriter (28s)
Starting test(python3.6): pyspark.sql.types
Finished test(python3.6): pyspark.sql.types (9s)
Starting test(python3.6): pyspark.sql.udf
Finished test(python3.6): pyspark.sql.streaming (18s)
Starting test(python3.6): pyspark.sql.window
Finished test(python3.6): pyspark.sql.udf (18s)
Starting test(python3.9): pyspark.mllib.classification
Finished test(python3.6): pyspark.sql.window (19s)
Starting test(python3.9): pyspark.mllib.clustering
Finished test(python3.9): pyspark.mllib.classification (59s)
Starting test(python3.9): pyspark.mllib.evaluation
Finished test(python3.9): pyspark.mllib.clustering (72s)
Starting test(python3.9): pyspark.mllib.feature
Finished test(python3.9): pyspark.mllib.evaluation (31s)
Starting test(python3.9): pyspark.mllib.fpm
Finished test(python3.9): pyspark.mllib.feature (36s)
Starting test(python3.9): pyspark.mllib.linalg.__init__
Finished test(python3.9): pyspark.mllib.linalg.__init__ (0s)
Starting test(python3.9): pyspark.mllib.linalg.distributed
Finished test(python3.9): pyspark.mllib.fpm (28s)
Starting test(python3.9): pyspark.mllib.random
Finished test(python3.9): pyspark.mllib.random (16s)
Starting test(python3.9): pyspark.mllib.recommendation
Finished test(python3.9): pyspark.mllib.linalg.distributed (77s)
Starting test(python3.9): pyspark.mllib.regression
Finished test(python3.9): pyspark.mllib.recommendation (59s)
Starting test(python3.9): pyspark.mllib.stat.KernelDensity
Finished test(python3.9): pyspark.mllib.stat.KernelDensity (0s)
Starting test(python3.9): pyspark.mllib.stat._statistics
Finished test(python3.9): pyspark.mllib.stat._statistics (24s)
Starting test(python3.9): pyspark.mllib.tree
Finished test(python3.9): pyspark.mllib.regression (38s)
Starting test(python3.9): pyspark.mllib.util
Finished test(python3.9): pyspark.mllib.tree (24s)
Starting test(python3.9): pyspark.resource.tests.test_resources
Finished test(python3.9): pyspark.resource.tests.test_resources (8s)
Starting test(python3.9): pyspark.sql.avro.functions
Finished test(python3.9): pyspark.mllib.util (25s)
Starting test(python3.9): pyspark.sql.catalog
Finished test(python3.9): pyspark.sql.avro.functions (16s)
Starting test(python3.9): pyspark.sql.column
Finished test(python3.9): pyspark.sql.catalog (16s)
Starting test(python3.9): pyspark.sql.conf
Finished test(python3.9): pyspark.sql.conf (8s)
Starting test(python3.9): pyspark.sql.context
Finished test(python3.9): pyspark.sql.column (30s)
Starting test(python3.9): pyspark.sql.dataframe
Finished test(python3.9): pyspark.sql.context (26s)
Starting test(python3.9): pyspark.sql.functions
Finished test(python3.9): pyspark.sql.dataframe (57s)
Starting test(python3.9): pyspark.sql.group
Finished test(python3.9): pyspark.sql.functions (73s)
Starting test(python3.9): pyspark.sql.pandas.conversion
Finished test(python3.9): pyspark.sql.group (28s)
Starting test(python3.9): pyspark.sql.pandas.group_ops
Finished test(python3.9): pyspark.sql.pandas.conversion (9s)
Starting test(python3.9): pyspark.sql.pandas.map_ops
Finished test(python3.9): pyspark.sql.pandas.group_ops (11s)
Starting test(python3.9): pyspark.sql.pandas.serializers
Finished test(python3.9): pyspark.sql.pandas.serializers (0s)
Starting test(python3.9): pyspark.sql.pandas.typehints
Finished test(python3.9): pyspark.sql.pandas.typehints (0s)
Starting test(python3.9): pyspark.sql.pandas.types
Finished test(python3.9): pyspark.sql.pandas.map_ops (10s)
Starting test(python3.9): pyspark.sql.pandas.utils
Finished test(python3.9): pyspark.sql.pandas.types (0s)
Starting test(python3.9): pyspark.sql.readwriter
Finished test(python3.9): pyspark.sql.pandas.utils (0s)
Starting test(python3.9): pyspark.sql.session
Finished test(python3.9): pyspark.sql.session (23s)
Starting test(python3.9): pyspark.sql.streaming
Finished test(python3.9): pyspark.sql.readwriter (28s)
Starting test(python3.9): pyspark.sql.types
Finished test(python3.9): pyspark.sql.types (9s)
Starting test(python3.9): pyspark.sql.udf
Finished test(python3.9): pyspark.sql.streaming (16s)
Starting test(python3.9): pyspark.sql.window
Finished test(python3.9): pyspark.sql.udf (17s)
Finished test(python3.9): pyspark.sql.window (17s)
Tests passed in 2827 seconds

========================================================================
Running PySpark packaging tests
========================================================================
Constructing virtual env for testing
Using conda virtual environments
Testing pip installation with python 3.6
Using /tmp/tmp.ej2FwCYNuN for virtualenv
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done