Texte vergleichen

Finde den Unterschied zwischen zwei Textdateien

Live-Editor

Gleiches ausblenden

Zeilenumbruch aus

Ansicht

Vergleichsgenauigkeit

Syntaxhervorhebung

Diffchecker Desktop Der sicherste Weg, Diffchecker zu nutzen. Hol dir die Desktop-App: Deine Diffs verlassen nie deinen Computer!Desktop holen

Build modules: pyspark-core, pyspark-streaming, pyspark-ml

Erstellt vor 5 JahrenDiff läuft nie ab

129 Entfernungen

Zeilen
Gesamt
Entfernt

Zeichen
Gesamt
Entfernt

Um diese Funktion weiterhin zu nutzen, aktualisiere auf Diffchecker Pro Preise anzeigen

353 Zeilen

139 Hinzufügungen

Zeilen
Gesamt
Hinzugefügt

Zeichen
Gesamt
Hinzugefügt

Um diese Funktion weiterhin zu nutzen, aktualisiere auf Diffchecker Pro Preise anzeigen

362 Zeilen

Before

After

https://github.com/apache/spark/runs/2959035601

https://github.com/Yikun/spark/runs/2968286012

[success] Total time: 244 s (04:04), completed Jul 2, 2021 1:57:47 AM

========================================================================

Running Spark unit tests

========================================================================

Running PySpark tests

========================================================================

'PYARROW_IGNORE_TIMEZONE' environment variable was not set. It is required to set this environment variable to '1' in both driver and executor sides if you use pyarrow>=2.0.0. pandas-on-Spark will set it for you but it does not work if there is a Spark context already launched.

Running PySpark tests. Output is in /__w/spark/spark/python/unit-tests.log

Will test against the following Python executables: ['python3.6', 'python3.9', 'pypy3']

Will test the following Python modules: ['pyspark-core', 'pyspark-streaming', 'pyspark-ml']

python3.6 python_implementation is CPython

python3.6 version is: Python 3.6.13

python3.9 python_implementation is CPython

python3.9 version is: Python 3.9.5

pypy3 python_implementation is PyPy

pypy3 version is: Python 3.7.10 (7.3.5+dfsg-1~ppa1~ubuntu20.04, May 23 2021, 14:57:07)

[PyPy 7.3.5 with GCC 9.3.0]

Starting test(pypy3): pyspark.streaming.tests.test_context

Starting test(pypy3): pyspark.streaming.tests.test_dstream

Starting test(pypy3): pyspark.streaming.tests.test_context

Finished test(pypy3): pyspark.streaming.tests.test_dstream (1s) ... 21 tests were skipped

Starting test(pypy3): pyspark.streaming.tests.test_kinesis

Finished test(pypy3): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped

Finished test(pypy3): pyspark.streaming.tests.test_kinesis (1s) ... 2 tests were skipped

Starting test(pypy3): pyspark.streaming.tests.test_listener

Finished test(pypy3): pyspark.streaming.tests.test_listener (13s)

Finished test(pypy3): pyspark.streaming.tests.test_listener (15s)

Starting test(pypy3): pyspark.tests.test_appsubmit

Finished test(pypy3): pyspark.streaming.tests.test_context (24s)

Finished test(pypy3): pyspark.streaming.tests.test_context (26s)

Starting test(pypy3): pyspark.tests.test_broadcast

Finished test(pypy3): pyspark.tests.test_broadcast (49s)

Finished test(pypy3): pyspark.tests.test_broadcast (58s)

Starting test(pypy3): pyspark.tests.test_conf

Finished test(pypy3): pyspark.tests.test_conf (23s)

Finished test(pypy3): pyspark.tests.test_conf (27s)

Starting test(pypy3): pyspark.tests.test_context

Finished test(pypy3): pyspark.tests.test_appsubmit (103s)

Finished test(pypy3): pyspark.tests.test_appsubmit (133s)

Starting test(pypy3): pyspark.tests.test_daemon

Finished test(pypy3): pyspark.tests.test_daemon (7s)

Finished test(pypy3): pyspark.tests.test_daemon (8s)

Starting test(pypy3): pyspark.tests.test_install_spark

Finished test(pypy3): pyspark.tests.test_context (63s)

Finished test(pypy3): pyspark.tests.test_install_spark (24s)

Starting test(pypy3): pyspark.tests.test_join

Finished test(pypy3): pyspark.tests.test_join (9s)

Finished test(pypy3): pyspark.tests.test_context (78s)

Starting test(pypy3): pyspark.tests.test_pin_thread

Finished test(pypy3): pyspark.tests.test_join (17s)

Starting test(pypy3): pyspark.tests.test_profiler

Finished test(pypy3): pyspark.tests.test_profiler (10s)

Finished test(pypy3): pyspark.tests.test_profiler (12s)

Starting test(pypy3): pyspark.tests.test_rdd

Finished test(pypy3): pyspark.tests.test_rdd (101s)

Finished test(pypy3): pyspark.tests.test_pin_thread (44s)

Starting test(pypy3): pyspark.tests.test_rddbarrier

Finished test(pypy3): pyspark.tests.test_rddbarrier (6s)

Finished test(pypy3): pyspark.tests.test_rddbarrier (15s)

Starting test(pypy3): pyspark.tests.test_readwrite

Finished test(pypy3): pyspark.tests.test_readwrite (18s)

Finished test(pypy3): pyspark.tests.test_readwrite (25s)

Starting test(pypy3): pyspark.tests.test_serializers

Finished test(pypy3): pyspark.tests.test_serializers (9s)

Finished test(pypy3): pyspark.tests.test_serializers (10s)

Starting test(pypy3): pyspark.tests.test_shuffle

Finished test(pypy3): pyspark.tests.test_shuffle (10s)

Finished test(pypy3): pyspark.tests.test_shuffle (21s)

Starting test(pypy3): pyspark.tests.test_taskcontext

Finished test(pypy3): pyspark.tests.test_install_spark (208s)

Finished test(pypy3): pyspark.tests.test_rdd (124s)

Starting test(pypy3): pyspark.tests.test_util

Finished test(pypy3): pyspark.tests.test_util (4s)

Finished test(pypy3): pyspark.tests.test_util (5s)

Starting test(pypy3): pyspark.tests.test_worker

Finished test(pypy3): pyspark.tests.test_worker (19s)

Finished test(pypy3): pyspark.tests.test_worker (21s)

Starting test(python3.6): pyspark.ml.tests.test_algorithms

Finished test(pypy3): pyspark.tests.test_taskcontext (76s)

Finished test(pypy3): pyspark.tests.test_taskcontext (88s)

Starting test(python3.6): pyspark.ml.tests.test_base

Finished test(python3.6): pyspark.ml.tests.test_base (18s)

Finished test(python3.6): pyspark.ml.tests.test_base (21s)

Starting test(python3.6): pyspark.ml.tests.test_evaluation

Finished test(python3.6): pyspark.ml.tests.test_evaluation (25s)

Finished test(python3.6): pyspark.ml.tests.test_evaluation (24s)

Starting test(python3.6): pyspark.ml.tests.test_feature

Finished test(python3.6): pyspark.ml.tests.test_algorithms (93s)

Finished test(python3.6): pyspark.ml.tests.test_algorithms (115s)

Starting test(python3.6): pyspark.ml.tests.test_image

Finished test(python3.6): pyspark.ml.tests.test_feature (45s)

Starting test(python3.6): pyspark.ml.tests.test_linalg

Finished test(python3.6): pyspark.ml.tests.test_image (18s)

Starting test(python3.6): pyspark.ml.tests.test_linalg

Finished test(python3.6): pyspark.ml.tests.test_feature (34s)

Starting test(python3.6): pyspark.ml.tests.test_param

Finished test(python3.6): pyspark.ml.tests.test_param (20s)

Finished test(python3.6): pyspark.ml.tests.test_param (23s)

Starting test(python3.6): pyspark.ml.tests.test_persistence

Finished test(python3.6): pyspark.ml.tests.test_linalg (36s)

Finished test(python3.6): pyspark.ml.tests.test_linalg (40s)

Starting test(python3.6): pyspark.ml.tests.test_pipeline

Finished test(python3.6): pyspark.ml.tests.test_pipeline (7s)

Starting test(python3.6): pyspark.ml.tests.test_stat

Finished test(python3.6): pyspark.ml.tests.test_stat (20s)

Finished test(python3.6): pyspark.ml.tests.test_stat (23s)

Starting test(python3.6): pyspark.ml.tests.test_training_summary

Finished test(python3.6): pyspark.ml.tests.test_persistence (80s)

Finished test(python3.6): pyspark.ml.tests.test_persistence (95s)

Starting test(python3.6): pyspark.ml.tests.test_tuning

Finished test(python3.6): pyspark.ml.tests.test_training_summary (50s)

Finished test(python3.6): pyspark.ml.tests.test_training_summary (59s)

Starting test(python3.6): pyspark.ml.tests.test_util

Finished test(python3.6): pyspark.ml.tests.test_util (29s)

Finished test(python3.6): pyspark.ml.tests.test_util (36s)

Starting test(python3.6): pyspark.ml.tests.test_wrapper

Finished test(python3.6): pyspark.ml.tests.test_wrapper (23s)

Finished test(python3.6): pyspark.ml.tests.test_wrapper (29s)

Starting test(python3.6): pyspark.streaming.tests.test_context

Finished test(python3.6): pyspark.streaming.tests.test_context (26s)

Finished test(python3.6): pyspark.streaming.tests.test_context (31s)

Starting test(python3.6): pyspark.streaming.tests.test_dstream

Finished test(python3.6): pyspark.streaming.tests.test_dstream (116s)

Finished test(python3.6): pyspark.streaming.tests.test_dstream (124s)

Starting test(python3.6): pyspark.streaming.tests.test_kinesis

Finished test(python3.6): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped

Starting test(python3.6): pyspark.streaming.tests.test_listener

Finished test(python3.6): pyspark.streaming.tests.test_listener (9s)

Finished test(python3.6): pyspark.streaming.tests.test_listener (14s)

Starting test(python3.6): pyspark.tests.test_appsubmit

Finished test(python3.6): pyspark.tests.test_appsubmit (97s)

Finished test(python3.6): pyspark.tests.test_appsubmit (122s)

Starting test(python3.6): pyspark.tests.test_broadcast

Finished test(python3.6): pyspark.ml.tests.test_tuning (341s)

Finished test(python3.6): pyspark.tests.test_broadcast (54s)

Starting test(python3.6): pyspark.tests.test_conf

Finished test(python3.6): pyspark.tests.test_broadcast (46s)

Finished test(python3.6): pyspark.ml.tests.test_tuning (422s)

Starting test(python3.6): pyspark.tests.test_context

Finished test(python3.6): pyspark.tests.test_conf (22s)

Finished test(python3.6): pyspark.tests.test_conf (16s)

Starting test(python3.6): pyspark.tests.test_daemon

Finished test(python3.6): pyspark.tests.test_daemon (5s)

Starting test(python3.6): pyspark.tests.test_install_spark

Finished test(python3.6): pyspark.tests.test_install_spark (17s)

Finished test(python3.6): pyspark.tests.test_install_spark (38s)

Starting test(python3.6): pyspark.tests.test_join

Finished test(python3.6): pyspark.tests.test_join (17s)

Finished test(python3.6): pyspark.tests.test_join (10s)

Starting test(python3.6): pyspark.tests.test_pin_thread

Finished test(python3.6): pyspark.tests.test_context (65s)

Starting test(python3.6): pyspark.tests.test_profiler

Finished test(python3.6): pyspark.tests.test_profiler (10s)

Finished test(python3.6): pyspark.tests.test_profiler (14s)

Starting test(python3.6): pyspark.tests.test_rdd

Finished test(python3.6): pyspark.tests.test_context (63s)

Finished test(python3.6): pyspark.tests.test_pin_thread (42s)

Starting test(python3.6): pyspark.tests.test_rddbarrier

Finished test(python3.6): pyspark.tests.test_rddbarrier (11s)

Finished test(python3.6): pyspark.tests.test_rddbarrier (6s)

Starting test(python3.6): pyspark.tests.test_readwrite

Finished test(python3.6): pyspark.tests.test_readwrite (25s)

Finished test(python3.6): pyspark.tests.test_readwrite (17s)

Starting test(python3.6): pyspark.tests.test_serializers

Finished test(python3.6): pyspark.tests.test_serializers (7s)

Finished test(python3.6): pyspark.tests.test_serializers (12s)

Starting test(python3.6): pyspark.tests.test_shuffle

Finished test(python3.6): pyspark.tests.test_shuffle (8s)

Finished test(python3.6): pyspark.tests.test_shuffle (12s)

Starting test(python3.6): pyspark.tests.test_taskcontext

Finished test(python3.6): pyspark.tests.test_rdd (99s)

Finished test(python3.6): pyspark.tests.test_rdd (96s)

Starting test(python3.6): pyspark.tests.test_util

Finished test(python3.6): pyspark.tests.test_util (8s)

Finished test(python3.6): pyspark.tests.test_util (5s)

Starting test(python3.6): pyspark.tests.test_worker

Finished test(python3.6): pyspark.tests.test_taskcontext (63s)

Finished test(python3.6): pyspark.tests.test_worker (20s)

Starting test(python3.9): pyspark.ml.tests.test_algorithms

Finished test(python3.6): pyspark.tests.test_worker (18s)

Finished test(python3.6): pyspark.tests.test_taskcontext (64s)

Starting test(python3.9): pyspark.ml.tests.test_base

Finished test(python3.9): pyspark.ml.tests.test_base (19s)

Finished test(python3.9): pyspark.ml.tests.test_base (20s)

Starting test(python3.9): pyspark.ml.tests.test_evaluation

Finished test(python3.9): pyspark.ml.tests.test_evaluation (22s)

Finished test(python3.9): pyspark.ml.tests.test_evaluation (25s)

Starting test(python3.9): pyspark.ml.tests.test_feature

Finished test(python3.9): pyspark.ml.tests.test_feature (38s)

Finished test(python3.9): pyspark.ml.tests.test_feature (43s)

Starting test(python3.9): pyspark.ml.tests.test_image

Finished test(python3.9): pyspark.ml.tests.test_algorithms (103s)

Finished test(python3.9): pyspark.ml.tests.test_algorithms (112s)

Starting test(python3.9): pyspark.ml.tests.test_linalg

Finished test(python3.9): pyspark.ml.tests.test_image (19s)

Finished test(python3.9): pyspark.ml.tests.test_image (18s)

Starting test(python3.9): pyspark.ml.tests.test_param

Finished test(python3.9): pyspark.ml.tests.test_param (21s)

Finished test(python3.9): pyspark.ml.tests.test_param (22s)

Starting test(python3.9): pyspark.ml.tests.test_persistence

Finished test(python3.9): pyspark.ml.tests.test_linalg (36s)

Finished test(python3.9): pyspark.ml.tests.test_linalg (39s)

Starting test(python3.9): pyspark.ml.tests.test_pipeline

Finished test(python3.9): pyspark.ml.tests.test_pipeline (7s)

Starting test(python3.9): pyspark.ml.tests.test_stat

Finished test(python3.9): pyspark.ml.tests.test_stat (20s)

Starting test(python3.9): pyspark.ml.tests.test_training_summary

Finished test(python3.9): pyspark.ml.tests.test_persistence (75s)

Finished test(python3.9): pyspark.ml.tests.test_training_summary (55s)

Starting test(python3.9): pyspark.ml.tests.test_tuning

Finished test(python3.9): pyspark.ml.tests.test_training_summary (50s)

Finished test(python3.9): pyspark.ml.tests.test_persistence (89s)

Starting test(python3.9): pyspark.ml.tests.test_util

Finished test(python3.9): pyspark.ml.tests.test_util (29s)

Finished test(python3.9): pyspark.ml.tests.test_util (33s)

Starting test(python3.9): pyspark.ml.tests.test_wrapper

Finished test(python3.9): pyspark.ml.tests.test_wrapper (24s)

Finished test(python3.9): pyspark.ml.tests.test_wrapper (25s)

Starting test(python3.9): pyspark.streaming.tests.test_context

Finished test(python3.9): pyspark.streaming.tests.test_context (25s)

Finished test(python3.9): pyspark.streaming.tests.test_context (28s)

Starting test(python3.9): pyspark.streaming.tests.test_dstream

Finished test(python3.9): pyspark.streaming.tests.test_dstream (116s)

Finished test(python3.9): pyspark.streaming.tests.test_dstream (120s)

Starting test(python3.9): pyspark.streaming.tests.test_kinesis

Finished test(python3.9): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped

Starting test(python3.9): pyspark.streaming.tests.test_listener

Finished test(python3.9): pyspark.streaming.tests.test_listener (9s)

Finished test(python3.9): pyspark.streaming.tests.test_listener (12s)

Starting test(python3.9): pyspark.tests.test_appsubmit

Finished test(python3.9): pyspark.tests.test_appsubmit (94s)

Finished test(python3.9): pyspark.tests.test_appsubmit (109s)

Starting test(python3.9): pyspark.tests.test_broadcast

Finished test(python3.9): pyspark.ml.tests.test_tuning (343s)

Finished test(python3.9): pyspark.ml.tests.test_tuning (376s)

Starting test(python3.9): pyspark.tests.test_conf

Finished test(python3.9): pyspark.tests.test_broadcast (44s)

Finished test(python3.9): pyspark.tests.test_broadcast (48s)

Starting test(python3.9): pyspark.tests.test_context

Finished test(python3.9): pyspark.tests.test_conf (21s)

Starting test(python3.9): pyspark.tests.test_daemon

Finished test(python3.9): pyspark.tests.test_daemon (5s)

Finished test(python3.9): pyspark.tests.test_daemon (4s)

Starting test(python3.9): pyspark.tests.test_install_spark

Finished test(python3.9): pyspark.tests.test_install_spark (19s)

Finished test(python3.9): pyspark.tests.test_install_spark (21s)

Starting test(python3.9): pyspark.tests.test_join

Finished test(python3.9): pyspark.tests.test_join (16s)

Finished test(python3.9): pyspark.tests.test_join (20s)

Starting test(python3.9): pyspark.tests.test_pin_thread

Finished test(python3.9): pyspark.tests.test_context (71s)

Starting test(python3.9): pyspark.tests.test_profiler

Finished test(python3.9): pyspark.tests.test_profiler (14s)

Finished test(python3.9): pyspark.tests.test_profiler (10s)

Starting test(python3.9): pyspark.tests.test_rdd

Finished test(python3.9): pyspark.tests.test_context (65s)

Finished test(python3.9): pyspark.tests.test_pin_thread (39s)

Starting test(python3.9): pyspark.tests.test_rddbarrier

Finished test(python3.9): pyspark.tests.test_rddbarrier (12s)

Finished test(python3.9): pyspark.tests.test_rddbarrier (9s)

Starting test(python3.9): pyspark.tests.test_readwrite

Finished test(python3.9): pyspark.tests.test_readwrite (22s)

Finished test(python3.9): pyspark.tests.test_readwrite (16s)

Starting test(python3.9): pyspark.tests.test_serializers

Finished test(python3.9): pyspark.tests.test_serializers (8s)

Starting test(python3.9): pyspark.tests.test_shuffle

Finished test(python3.9): pyspark.tests.test_shuffle (7s)

Finished test(python3.9): pyspark.tests.test_shuffle (13s)

Starting test(python3.9): pyspark.tests.test_taskcontext

Finished test(python3.9): pyspark.tests.test_rdd (98s)

Finished test(python3.9): pyspark.tests.test_rdd (95s)

Starting test(python3.9): pyspark.tests.test_util

Finished test(python3.9): pyspark.tests.test_util (8s)

Finished test(python3.9): pyspark.tests.test_util (4s)

Starting test(python3.9): pyspark.tests.test_worker

Finished test(python3.9): pyspark.tests.test_worker (17s)

Finished test(python3.9): pyspark.tests.test_worker (19s)

Starting test(pypy3): pyspark.accumulators

Finished test(python3.9): pyspark.tests.test_taskcontext (68s)

Finished test(pypy3): pyspark.accumulators (12s)

Starting test(pypy3): pyspark.broadcast

Finished test(pypy3): pyspark.accumulators (10s)

Finished test(python3.9): pyspark.tests.test_taskcontext (70s)

Starting test(pypy3): pyspark.conf

Finished test(pypy3): pyspark.broadcast (9s)

Finished test(pypy3): pyspark.broadcast (7s)

Starting test(pypy3): pyspark.context

Finished test(pypy3): pyspark.conf (5s)

Finished test(pypy3): pyspark.conf (7s)

Starting test(pypy3): pyspark.profiler

Finished test(pypy3): pyspark.profiler (11s)

Finished test(pypy3): pyspark.profiler (13s)

Starting test(pypy3): pyspark.rdd

Finished test(pypy3): pyspark.context (27s)

Finished test(pypy3): pyspark.context (30s)

Starting test(pypy3): pyspark.serializers

Finished test(pypy3): pyspark.serializers (21s)

Finished test(pypy3): pyspark.serializers (25s)

Starting test(pypy3): pyspark.shuffle

Finished test(pypy3): pyspark.shuffle (2s)

Finished test(pypy3): pyspark.shuffle (3s)

Starting test(pypy3): pyspark.streaming.util

Finished test(pypy3): pyspark.streaming.util (1s)

Starting test(pypy3): pyspark.util

Finished test(pypy3): pyspark.util (1s)

Starting test(python3.6): pyspark.accumulators

Finished test(pypy3): pyspark.rdd (47s)

Finished test(pypy3): pyspark.rdd (52s)

Starting test(python3.6): pyspark.broadcast

Finished test(python3.6): pyspark.accumulators (13s)

Finished test(python3.6): pyspark.accumulators (14s)

Starting test(python3.6): pyspark.conf

Finished test(python3.6): pyspark.broadcast (7s)

Finished test(python3.6): pyspark.broadcast (9s)

Starting test(python3.6): pyspark.context

Finished test(python3.6): pyspark.conf (6s)

Starting test(python3.6): pyspark.ml.classification

Finished test(python3.6): pyspark.context (26s)

Finished test(python3.6): pyspark.context (29s)

Starting test(python3.6): pyspark.ml.clustering

Finished test(python3.6): pyspark.ml.clustering (62s)

Finished test(python3.6): pyspark.ml.clustering (73s)

Starting test(python3.6): pyspark.ml.evaluation

Finished test(python3.6): pyspark.ml.classification (107s)

Finished test(python3.6): pyspark.ml.classification (119s)

Starting test(python3.6): pyspark.ml.feature

Finished test(python3.6): pyspark.ml.evaluation (31s)

Finished test(python3.6): pyspark.ml.evaluation (33s)

Starting test(python3.6): pyspark.ml.fpm

Finished test(python3.6): pyspark.ml.fpm (31s)

Finished test(python3.6): pyspark.ml.fpm (36s)

Starting test(python3.6): pyspark.ml.functions

Finished test(python3.6): pyspark.ml.functions (22s)

Finished test(python3.6): pyspark.ml.functions (24s)

Starting test(python3.6): pyspark.ml.image

Finished test(python3.6): pyspark.ml.image (9s)

Starting test(python3.6): pyspark.ml.linalg.__init__

Finished test(python3.6): pyspark.ml.linalg.__init__ (0s)

Starting test(python3.6): pyspark.ml.recommendation

Finished test(python3.6): pyspark.ml.feature (102s)

Finished test(python3.6): pyspark.ml.feature (107s)

Starting test(python3.6): pyspark.ml.regression

Finished test(python3.6): pyspark.ml.recommendation (48s)

Finished test(python3.6): pyspark.ml.recommendation (53s)

Starting test(python3.6): pyspark.ml.stat

Finished test(python3.6): pyspark.ml.stat (32s)

Finished test(python3.6): pyspark.ml.stat (37s)

Starting test(python3.6): pyspark.ml.tuning

Finished test(python3.6): pyspark.ml.regression (63s)

Finished test(python3.6): pyspark.ml.regression (73s)

Starting test(python3.6): pyspark.profiler

Finished test(python3.6): pyspark.profiler (10s)

Finished test(python3.6): pyspark.profiler (11s)

Starting test(python3.6): pyspark.rdd

Finished test(python3.6): pyspark.ml.tuning (44s)

Finished test(python3.6): pyspark.ml.tuning (52s)

Starting test(python3.6): pyspark.serializers

Finished test(python3.6): pyspark.serializers (14s)

Finished test(python3.6): pyspark.rdd (48s)

Starting test(python3.6): pyspark.shuffle

Finished test(python3.6): pyspark.shuffle (0s)

Starting test(python3.6): pyspark.streaming.util

Finished test(python3.6): pyspark.streaming.util (0s)

Starting test(python3.6): pyspark.util

Finished test(python3.6): pyspark.util (0s)

Starting test(python3.9): pyspark.accumulators

Finished test(python3.6): pyspark.rdd (45s)

Finished test(python3.6): pyspark.serializers (13s)

Starting test(python3.9): pyspark.broadcast

Finished test(python3.9): pyspark.accumulators (10s)

Starting test(python3.9): pyspark.conf

Finished test(python3.9): pyspark.broadcast (8s)

Finished test(python3.9): pyspark.broadcast (9s)

Starting test(python3.9): pyspark.context

Finished test(python3.9): pyspark.conf (6s)

Starting test(python3.9): pyspark.ml.classification

Finished test(python3.9): pyspark.context (26s)

Finished test(python3.9): pyspark.context (28s)

Starting test(python3.9): pyspark.ml.clustering

Finished test(python3.9): pyspark.ml.clustering (62s)

Finished test(python3.9): pyspark.ml.clustering (68s)

Starting test(python3.9): pyspark.ml.evaluation

Finished test(python3.9): pyspark.ml.classification (105s)

Finished test(python3.9): pyspark.ml.classification (116s)

Starting test(python3.9): pyspark.ml.feature

Finished test(python3.9): pyspark.ml.evaluation (29s)

Finished test(python3.9): pyspark.ml.evaluation (33s)

Starting test(python3.9): pyspark.ml.fpm

Finished test(python3.9): pyspark.ml.fpm (31s)

Finished test(python3.9): pyspark.ml.fpm (34s)

Starting test(python3.9): pyspark.ml.functions

Finished test(python3.9): pyspark.ml.functions (22s)

Finished test(python3.9): pyspark.ml.functions (24s)

Starting test(python3.9): pyspark.ml.image

Finished test(python3.9): pyspark.ml.image (7s)

Finished test(python3.9): pyspark.ml.image (9s)

Starting test(python3.9): pyspark.ml.linalg.__init__

Finished test(python3.9): pyspark.ml.linalg.__init__ (0s)

Starting test(python3.9): pyspark.ml.recommendation

Finished test(python3.9): pyspark.ml.feature (102s)

Finished test(python3.9): pyspark.ml.feature (114s)

Starting test(python3.9): pyspark.ml.regression

Finished test(python3.9): pyspark.ml.recommendation (44s)

Finished test(python3.9): pyspark.ml.recommendation (51s)

Starting test(python3.9): pyspark.ml.stat

Finished test(python3.9): pyspark.ml.stat (33s)

Finished test(python3.9): pyspark.ml.stat (40s)

Starting test(python3.9): pyspark.ml.tuning

Finished test(python3.9): pyspark.ml.regression (60s)

Finished test(python3.9): pyspark.ml.regression (72s)

Starting test(python3.9): pyspark.profiler

Finished test(python3.9): pyspark.profiler (11s)

Finished test(python3.9): pyspark.profiler (13s)

Starting test(python3.9): pyspark.rdd

Finished test(python3.9): pyspark.ml.tuning (43s)

Finished test(python3.9): pyspark.ml.tuning (51s)

Starting test(python3.9): pyspark.serializers

Finished test(python3.9): pyspark.serializers (13s)

Finished test(python3.9): pyspark.serializers (14s)

Starting test(python3.9): pyspark.shuffle

Finished test(python3.9): pyspark.shuffle (0s)

Starting test(python3.9): pyspark.streaming.util

Finished test(python3.9): pyspark.streaming.util (0s)

Starting test(python3.9): pyspark.util

Finished test(python3.9): pyspark.util (6s)

Finished test(python3.9): pyspark.util (8s)

Finished test(python3.9): pyspark.rdd (42s)

Finished test(python3.9): pyspark.rdd (45s)

Tests passed in 2611 seconds

Tests passed in 2908 seconds

Skipped tests in pyspark.streaming.tests.test_dstream with pypy3:

test_cogroup (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_failed_func (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_failed_func2 (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_failed_func_with_reseting_failure (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_full_outer_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_left_outer_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_repartition (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_right_outer_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_union (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_update_state_by_key (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_update_state_by_key_initial_rdd (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_get_or_create_and_get_active_or_create (pyspark.streaming.tests.test_dstream.CheckpointTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_transform_function_serializer_failure (pyspark.streaming.tests.test_dstream.CheckpointTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_count_by_value_and_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_count_by_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_count_by_window_large (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_group_by_key_and_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_reduce_by_invalid_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_reduce_by_key_and_window_with_none_invFunc (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

test_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

Skipped tests in pyspark.streaming.tests.test_kinesis with pypy3:

test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."

test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."

Skipped tests in pyspark.streaming.tests.test_kinesis with python3.6:

test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)

test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)

Skipped tests in pyspark.streaming.tests.test_kinesis with python3.9:

test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)

test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)

========================================================================

Running PySpark packaging tests

========================================================================

Constructing virtual env for testing

Using conda virtual environments

Testing pip installation with python 3.6

Using /tmp/tmp.dkSnux7voE for virtualenv

Using /tmp/tmp.MMTZT5H3SO for virtualenv

Collecting package metadata (current_repodata.json): ...working... done

Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.

Collecting package metadata (repodata.json): ...working... done

Solving environment: ...working... done

Gespeicherte Diffs

Originaltext

Datei öffnen

Before
https://github.com/apache/spark/runs/2959035601

========================================================================
Running Spark unit tests
========================================================================

========================================================================
Running PySpark tests
========================================================================
Running PySpark tests. Output is in /__w/spark/spark/python/unit-tests.log
Will test against the following Python executables: ['python3.6', 'python3.9', 'pypy3']
Will test the following Python modules: ['pyspark-core', 'pyspark-streaming', 'pyspark-ml']
python3.6 python_implementation is CPython
python3.6 version is: Python 3.6.13
python3.9 python_implementation is CPython
python3.9 version is: Python 3.9.5
pypy3 python_implementation is PyPy
pypy3 version is: Python 3.7.10 (7.3.5+dfsg-1~ppa1~ubuntu20.04, May 23 2021, 14:57:07)
[PyPy 7.3.5 with GCC 9.3.0]
Starting test(pypy3): pyspark.streaming.tests.test_context
Starting test(pypy3): pyspark.streaming.tests.test_dstream
Finished test(pypy3): pyspark.streaming.tests.test_dstream (1s) ... 21 tests were skipped
Starting test(pypy3): pyspark.streaming.tests.test_kinesis
Finished test(pypy3): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(pypy3): pyspark.streaming.tests.test_listener
Finished test(pypy3): pyspark.streaming.tests.test_listener (13s)
Starting test(pypy3): pyspark.tests.test_appsubmit
Finished test(pypy3): pyspark.streaming.tests.test_context (24s)
Starting test(pypy3): pyspark.tests.test_broadcast
Finished test(pypy3): pyspark.tests.test_broadcast (49s)
Starting test(pypy3): pyspark.tests.test_conf
Finished test(pypy3): pyspark.tests.test_conf (23s)
Starting test(pypy3): pyspark.tests.test_context
Finished test(pypy3): pyspark.tests.test_appsubmit (103s)
Starting test(pypy3): pyspark.tests.test_daemon
Finished test(pypy3): pyspark.tests.test_daemon (7s)
Starting test(pypy3): pyspark.tests.test_install_spark
Finished test(pypy3): pyspark.tests.test_context (63s)
Starting test(pypy3): pyspark.tests.test_join
Finished test(pypy3): pyspark.tests.test_join (9s)
Starting test(pypy3): pyspark.tests.test_profiler
Finished test(pypy3): pyspark.tests.test_profiler (10s)
Starting test(pypy3): pyspark.tests.test_rdd
Finished test(pypy3): pyspark.tests.test_rdd (101s)
Starting test(pypy3): pyspark.tests.test_rddbarrier
Finished test(pypy3): pyspark.tests.test_rddbarrier (6s)
Starting test(pypy3): pyspark.tests.test_readwrite
Finished test(pypy3): pyspark.tests.test_readwrite (18s)
Starting test(pypy3): pyspark.tests.test_serializers
Finished test(pypy3): pyspark.tests.test_serializers (9s)
Starting test(pypy3): pyspark.tests.test_shuffle
Finished test(pypy3): pyspark.tests.test_shuffle (10s)
Starting test(pypy3): pyspark.tests.test_taskcontext
Finished test(pypy3): pyspark.tests.test_install_spark (208s)
Starting test(pypy3): pyspark.tests.test_util
Finished test(pypy3): pyspark.tests.test_util (4s)
Starting test(pypy3): pyspark.tests.test_worker
Finished test(pypy3): pyspark.tests.test_worker (19s)
Starting test(python3.6): pyspark.ml.tests.test_algorithms
Finished test(pypy3): pyspark.tests.test_taskcontext (76s)
Starting test(python3.6): pyspark.ml.tests.test_base
Finished test(python3.6): pyspark.ml.tests.test_base (18s)
Starting test(python3.6): pyspark.ml.tests.test_evaluation
Finished test(python3.6): pyspark.ml.tests.test_evaluation (25s)
Starting test(python3.6): pyspark.ml.tests.test_feature
Finished test(python3.6): pyspark.ml.tests.test_algorithms (93s)
Starting test(python3.6): pyspark.ml.tests.test_image
Finished test(python3.6): pyspark.ml.tests.test_image (18s)
Starting test(python3.6): pyspark.ml.tests.test_linalg
Finished test(python3.6): pyspark.ml.tests.test_feature (34s)
Starting test(python3.6): pyspark.ml.tests.test_param
Finished test(python3.6): pyspark.ml.tests.test_param (20s)
Starting test(python3.6): pyspark.ml.tests.test_persistence
Finished test(python3.6): pyspark.ml.tests.test_linalg (36s)
Starting test(python3.6): pyspark.ml.tests.test_pipeline
Finished test(python3.6): pyspark.ml.tests.test_pipeline (7s)
Starting test(python3.6): pyspark.ml.tests.test_stat
Finished test(python3.6): pyspark.ml.tests.test_stat (20s)
Starting test(python3.6): pyspark.ml.tests.test_training_summary
Finished test(python3.6): pyspark.ml.tests.test_persistence (80s)
Starting test(python3.6): pyspark.ml.tests.test_tuning
Finished test(python3.6): pyspark.ml.tests.test_training_summary (50s)
Starting test(python3.6): pyspark.ml.tests.test_util
Finished test(python3.6): pyspark.ml.tests.test_util (29s)
Starting test(python3.6): pyspark.ml.tests.test_wrapper
Finished test(python3.6): pyspark.ml.tests.test_wrapper (23s)
Starting test(python3.6): pyspark.streaming.tests.test_context
Finished test(python3.6): pyspark.streaming.tests.test_context (26s)
Starting test(python3.6): pyspark.streaming.tests.test_dstream
Finished test(python3.6): pyspark.streaming.tests.test_dstream (116s)
Starting test(python3.6): pyspark.streaming.tests.test_kinesis
Finished test(python3.6): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(python3.6): pyspark.streaming.tests.test_listener
Finished test(python3.6): pyspark.streaming.tests.test_listener (9s)
Starting test(python3.6): pyspark.tests.test_appsubmit
Finished test(python3.6): pyspark.tests.test_appsubmit (97s)
Starting test(python3.6): pyspark.tests.test_broadcast
Finished test(python3.6): pyspark.ml.tests.test_tuning (341s)
Starting test(python3.6): pyspark.tests.test_conf
Finished test(python3.6): pyspark.tests.test_broadcast (46s)
Starting test(python3.6): pyspark.tests.test_context
Finished test(python3.6): pyspark.tests.test_conf (22s)
Starting test(python3.6): pyspark.tests.test_daemon
Finished test(python3.6): pyspark.tests.test_daemon (5s)
Starting test(python3.6): pyspark.tests.test_install_spark
Finished test(python3.6): pyspark.tests.test_install_spark (17s)
Starting test(python3.6): pyspark.tests.test_join
Finished test(python3.6): pyspark.tests.test_join (17s)
Starting test(python3.6): pyspark.tests.test_profiler
Finished test(python3.6): pyspark.tests.test_profiler (10s)
Starting test(python3.6): pyspark.tests.test_rdd
Finished test(python3.6): pyspark.tests.test_context (63s)
Starting test(python3.6): pyspark.tests.test_rddbarrier
Finished test(python3.6): pyspark.tests.test_rddbarrier (11s)
Starting test(python3.6): pyspark.tests.test_readwrite
Finished test(python3.6): pyspark.tests.test_readwrite (25s)
Starting test(python3.6): pyspark.tests.test_serializers
Finished test(python3.6): pyspark.tests.test_serializers (7s)
Starting test(python3.6): pyspark.tests.test_shuffle
Finished test(python3.6): pyspark.tests.test_shuffle (8s)
Starting test(python3.6): pyspark.tests.test_taskcontext
Finished test(python3.6): pyspark.tests.test_rdd (99s)
Starting test(python3.6): pyspark.tests.test_util
Finished test(python3.6): pyspark.tests.test_util (8s)
Starting test(python3.6): pyspark.tests.test_worker
Finished test(python3.6): pyspark.tests.test_taskcontext (63s)
Starting test(python3.9): pyspark.ml.tests.test_algorithms
Finished test(python3.6): pyspark.tests.test_worker (18s)
Starting test(python3.9): pyspark.ml.tests.test_base
Finished test(python3.9): pyspark.ml.tests.test_base (19s)
Starting test(python3.9): pyspark.ml.tests.test_evaluation
Finished test(python3.9): pyspark.ml.tests.test_evaluation (22s)
Starting test(python3.9): pyspark.ml.tests.test_feature
Finished test(python3.9): pyspark.ml.tests.test_feature (38s)
Starting test(python3.9): pyspark.ml.tests.test_image
Finished test(python3.9): pyspark.ml.tests.test_algorithms (103s)
Starting test(python3.9): pyspark.ml.tests.test_linalg
Finished test(python3.9): pyspark.ml.tests.test_image (19s)
Starting test(python3.9): pyspark.ml.tests.test_param
Finished test(python3.9): pyspark.ml.tests.test_param (21s)
Starting test(python3.9): pyspark.ml.tests.test_persistence
Finished test(python3.9): pyspark.ml.tests.test_linalg (36s)
Starting test(python3.9): pyspark.ml.tests.test_pipeline
Finished test(python3.9): pyspark.ml.tests.test_pipeline (7s)
Starting test(python3.9): pyspark.ml.tests.test_stat
Finished test(python3.9): pyspark.ml.tests.test_stat (20s)
Starting test(python3.9): pyspark.ml.tests.test_training_summary
Finished test(python3.9): pyspark.ml.tests.test_persistence (75s)
Starting test(python3.9): pyspark.ml.tests.test_tuning
Finished test(python3.9): pyspark.ml.tests.test_training_summary (50s)
Starting test(python3.9): pyspark.ml.tests.test_util
Finished test(python3.9): pyspark.ml.tests.test_util (29s)
Starting test(python3.9): pyspark.ml.tests.test_wrapper
Finished test(python3.9): pyspark.ml.tests.test_wrapper (24s)
Starting test(python3.9): pyspark.streaming.tests.test_context
Finished test(python3.9): pyspark.streaming.tests.test_context (25s)
Starting test(python3.9): pyspark.streaming.tests.test_dstream
Finished test(python3.9): pyspark.streaming.tests.test_dstream (116s)
Starting test(python3.9): pyspark.streaming.tests.test_kinesis
Finished test(python3.9): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(python3.9): pyspark.streaming.tests.test_listener
Finished test(python3.9): pyspark.streaming.tests.test_listener (9s)
Starting test(python3.9): pyspark.tests.test_appsubmit
Finished test(python3.9): pyspark.tests.test_appsubmit (94s)
Starting test(python3.9): pyspark.tests.test_broadcast
Finished test(python3.9): pyspark.ml.tests.test_tuning (343s)
Starting test(python3.9): pyspark.tests.test_conf
Finished test(python3.9): pyspark.tests.test_broadcast (44s)
Starting test(python3.9): pyspark.tests.test_context
Finished test(python3.9): pyspark.tests.test_conf (21s)
Starting test(python3.9): pyspark.tests.test_daemon
Finished test(python3.9): pyspark.tests.test_daemon (5s)
Starting test(python3.9): pyspark.tests.test_install_spark
Finished test(python3.9): pyspark.tests.test_install_spark (19s)
Starting test(python3.9): pyspark.tests.test_join
Finished test(python3.9): pyspark.tests.test_join (16s)
Starting test(python3.9): pyspark.tests.test_profiler
Finished test(python3.9): pyspark.tests.test_profiler (14s)
Starting test(python3.9): pyspark.tests.test_rdd
Finished test(python3.9): pyspark.tests.test_context (65s)
Starting test(python3.9): pyspark.tests.test_rddbarrier
Finished test(python3.9): pyspark.tests.test_rddbarrier (12s)
Starting test(python3.9): pyspark.tests.test_readwrite
Finished test(python3.9): pyspark.tests.test_readwrite (22s)
Starting test(python3.9): pyspark.tests.test_serializers
Finished test(python3.9): pyspark.tests.test_serializers (8s)
Starting test(python3.9): pyspark.tests.test_shuffle
Finished test(python3.9): pyspark.tests.test_shuffle (7s)
Starting test(python3.9): pyspark.tests.test_taskcontext
Finished test(python3.9): pyspark.tests.test_rdd (98s)
Starting test(python3.9): pyspark.tests.test_util
Finished test(python3.9): pyspark.tests.test_util (8s)
Starting test(python3.9): pyspark.tests.test_worker
Finished test(python3.9): pyspark.tests.test_worker (17s)
Starting test(pypy3): pyspark.accumulators
Finished test(python3.9): pyspark.tests.test_taskcontext (68s)
Starting test(pypy3): pyspark.broadcast
Finished test(pypy3): pyspark.accumulators (10s)
Starting test(pypy3): pyspark.conf
Finished test(pypy3): pyspark.broadcast (9s)
Starting test(pypy3): pyspark.context
Finished test(pypy3): pyspark.conf (5s)
Starting test(pypy3): pyspark.profiler
Finished test(pypy3): pyspark.profiler (11s)
Starting test(pypy3): pyspark.rdd
Finished test(pypy3): pyspark.context (27s)
Starting test(pypy3): pyspark.serializers
Finished test(pypy3): pyspark.serializers (21s)
Starting test(pypy3): pyspark.shuffle
Finished test(pypy3): pyspark.shuffle (2s)
Starting test(pypy3): pyspark.streaming.util
Finished test(pypy3): pyspark.streaming.util (1s)
Starting test(pypy3): pyspark.util
Finished test(pypy3): pyspark.util (1s)
Starting test(python3.6): pyspark.accumulators
Finished test(pypy3): pyspark.rdd (47s)
Starting test(python3.6): pyspark.broadcast
Finished test(python3.6): pyspark.accumulators (13s)
Starting test(python3.6): pyspark.conf
Finished test(python3.6): pyspark.broadcast (7s)
Starting test(python3.6): pyspark.context
Finished test(python3.6): pyspark.conf (6s)
Starting test(python3.6): pyspark.ml.classification
Finished test(python3.6): pyspark.context (26s)
Starting test(python3.6): pyspark.ml.clustering
Finished test(python3.6): pyspark.ml.clustering (62s)
Starting test(python3.6): pyspark.ml.evaluation
Finished test(python3.6): pyspark.ml.classification (107s)
Starting test(python3.6): pyspark.ml.feature
Finished test(python3.6): pyspark.ml.evaluation (31s)
Starting test(python3.6): pyspark.ml.fpm
Finished test(python3.6): pyspark.ml.fpm (31s)
Starting test(python3.6): pyspark.ml.functions
Finished test(python3.6): pyspark.ml.functions (22s)
Starting test(python3.6): pyspark.ml.image
Finished test(python3.6): pyspark.ml.image (9s)
Starting test(python3.6): pyspark.ml.linalg.__init__
Finished test(python3.6): pyspark.ml.linalg.__init__ (0s)
Starting test(python3.6): pyspark.ml.recommendation
Finished test(python3.6): pyspark.ml.feature (102s)
Starting test(python3.6): pyspark.ml.regression
Finished test(python3.6): pyspark.ml.recommendation (48s)
Starting test(python3.6): pyspark.ml.stat
Finished test(python3.6): pyspark.ml.stat (32s)
Starting test(python3.6): pyspark.ml.tuning
Finished test(python3.6): pyspark.ml.regression (63s)
Starting test(python3.6): pyspark.profiler
Finished test(python3.6): pyspark.profiler (10s)
Starting test(python3.6): pyspark.rdd
Finished test(python3.6): pyspark.ml.tuning (44s)
Starting test(python3.6): pyspark.serializers
Finished test(python3.6): pyspark.serializers (14s)
Starting test(python3.6): pyspark.shuffle
Finished test(python3.6): pyspark.shuffle (0s)
Starting test(python3.6): pyspark.streaming.util
Finished test(python3.6): pyspark.streaming.util (0s)
Starting test(python3.6): pyspark.util
Finished test(python3.6): pyspark.util (0s)
Starting test(python3.9): pyspark.accumulators
Finished test(python3.6): pyspark.rdd (45s)
Starting test(python3.9): pyspark.broadcast
Finished test(python3.9): pyspark.accumulators (10s)
Starting test(python3.9): pyspark.conf
Finished test(python3.9): pyspark.broadcast (8s)
Starting test(python3.9): pyspark.context
Finished test(python3.9): pyspark.conf (6s)
Starting test(python3.9): pyspark.ml.classification
Finished test(python3.9): pyspark.context (26s)
Starting test(python3.9): pyspark.ml.clustering
Finished test(python3.9): pyspark.ml.clustering (62s)
Starting test(python3.9): pyspark.ml.evaluation
Finished test(python3.9): pyspark.ml.classification (105s)
Starting test(python3.9): pyspark.ml.feature
Finished test(python3.9): pyspark.ml.evaluation (29s)
Starting test(python3.9): pyspark.ml.fpm
Finished test(python3.9): pyspark.ml.fpm (31s)
Starting test(python3.9): pyspark.ml.functions
Finished test(python3.9): pyspark.ml.functions (22s)
Starting test(python3.9): pyspark.ml.image
Finished test(python3.9): pyspark.ml.image (7s)
Starting test(python3.9): pyspark.ml.linalg.__init__
Finished test(python3.9): pyspark.ml.linalg.__init__ (0s)
Starting test(python3.9): pyspark.ml.recommendation
Finished test(python3.9): pyspark.ml.feature (102s)
Starting test(python3.9): pyspark.ml.regression
Finished test(python3.9): pyspark.ml.recommendation (44s)
Starting test(python3.9): pyspark.ml.stat
Finished test(python3.9): pyspark.ml.stat (33s)
Starting test(python3.9): pyspark.ml.tuning
Finished test(python3.9): pyspark.ml.regression (60s)
Starting test(python3.9): pyspark.profiler
Finished test(python3.9): pyspark.profiler (11s)
Starting test(python3.9): pyspark.rdd
Finished test(python3.9): pyspark.ml.tuning (43s)
Starting test(python3.9): pyspark.serializers
Finished test(python3.9): pyspark.serializers (13s)
Starting test(python3.9): pyspark.shuffle
Finished test(python3.9): pyspark.shuffle (0s)
Starting test(python3.9): pyspark.streaming.util
Finished test(python3.9): pyspark.streaming.util (0s)
Starting test(python3.9): pyspark.util
Finished test(python3.9): pyspark.util (6s)
Finished test(python3.9): pyspark.rdd (42s)
Tests passed in 2611 seconds

Skipped tests in pyspark.streaming.tests.test_dstream with pypy3:
    test_cogroup (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_failed_func (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_failed_func2 (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_failed_func_with_reseting_failure (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_full_outer_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_left_outer_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_repartition (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_right_outer_join (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_union (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_update_state_by_key (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_update_state_by_key_initial_rdd (pyspark.streaming.tests.test_dstream.BasicOperationTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_get_or_create_and_get_active_or_create (pyspark.streaming.tests.test_dstream.CheckpointTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_transform_function_serializer_failure (pyspark.streaming.tests.test_dstream.CheckpointTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_count_by_value_and_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_count_by_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_count_by_window_large (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_group_by_key_and_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_reduce_by_invalid_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_reduce_by_key_and_window_with_none_invFunc (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'
    test_window (pyspark.streaming.tests.test_dstream.WindowFunctionTests) ... skipped 'The tests fail in PyPy3 implementation for an unknown reason. With PyPy, it causes to hang DStream tests forever when Coverage report is used.'

Skipped tests in pyspark.streaming.tests.test_kinesis with pypy3:
    test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."
    test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."

Skipped tests in pyspark.streaming.tests.test_kinesis with python3.6:
      test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)
      test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)

Skipped tests in pyspark.streaming.tests.test_kinesis with python3.9:
      test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)
      test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... SKIP (0.000s)

========================================================================
Running PySpark packaging tests
========================================================================
Constructing virtual env for testing
Using conda virtual environments
Testing pip installation with python 3.6
Using /tmp/tmp.dkSnux7voE for virtualenv
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

Bearbeitung

Datei öffnen

After
https://github.com/Yikun/spark/runs/2968286012

[success] Total time: 244 s (04:04), completed Jul 2, 2021 1:57:47 AM

========================================================================
Running Spark unit tests
========================================================================

========================================================================
Running PySpark tests
========================================================================
'PYARROW_IGNORE_TIMEZONE' environment variable was not set. It is required to set this environment variable to '1' in both driver and executor sides if you use pyarrow>=2.0.0. pandas-on-Spark will set it for you but it does not work if there is a Spark context already launched.
Running PySpark tests. Output is in /__w/spark/spark/python/unit-tests.log
Will test against the following Python executables: ['python3.6', 'python3.9', 'pypy3']
Will test the following Python modules: ['pyspark-core', 'pyspark-streaming', 'pyspark-ml']
python3.6 python_implementation is CPython
python3.6 version is: Python 3.6.13
python3.9 python_implementation is CPython
python3.9 version is: Python 3.9.5
pypy3 python_implementation is PyPy
pypy3 version is: Python 3.7.10 (7.3.5+dfsg-1~ppa1~ubuntu20.04, May 23 2021, 14:57:07)
[PyPy 7.3.5 with GCC 9.3.0]
Starting test(pypy3): pyspark.streaming.tests.test_dstream
Starting test(pypy3): pyspark.streaming.tests.test_context
Finished test(pypy3): pyspark.streaming.tests.test_dstream (1s) ... 21 tests were skipped
Starting test(pypy3): pyspark.streaming.tests.test_kinesis
Finished test(pypy3): pyspark.streaming.tests.test_kinesis (1s) ... 2 tests were skipped
Starting test(pypy3): pyspark.streaming.tests.test_listener
Finished test(pypy3): pyspark.streaming.tests.test_listener (15s)
Starting test(pypy3): pyspark.tests.test_appsubmit
Finished test(pypy3): pyspark.streaming.tests.test_context (26s)
Starting test(pypy3): pyspark.tests.test_broadcast
Finished test(pypy3): pyspark.tests.test_broadcast (58s)
Starting test(pypy3): pyspark.tests.test_conf
Finished test(pypy3): pyspark.tests.test_conf (27s)
Starting test(pypy3): pyspark.tests.test_context
Finished test(pypy3): pyspark.tests.test_appsubmit (133s)
Starting test(pypy3): pyspark.tests.test_daemon
Finished test(pypy3): pyspark.tests.test_daemon (8s)
Starting test(pypy3): pyspark.tests.test_install_spark
Finished test(pypy3): pyspark.tests.test_install_spark (24s)
Starting test(pypy3): pyspark.tests.test_join
Finished test(pypy3): pyspark.tests.test_context (78s)
Starting test(pypy3): pyspark.tests.test_pin_thread
Finished test(pypy3): pyspark.tests.test_join (17s)
Starting test(pypy3): pyspark.tests.test_profiler
Finished test(pypy3): pyspark.tests.test_profiler (12s)
Starting test(pypy3): pyspark.tests.test_rdd
Finished test(pypy3): pyspark.tests.test_pin_thread (44s)
Starting test(pypy3): pyspark.tests.test_rddbarrier
Finished test(pypy3): pyspark.tests.test_rddbarrier (15s)
Starting test(pypy3): pyspark.tests.test_readwrite
Finished test(pypy3): pyspark.tests.test_readwrite (25s)
Starting test(pypy3): pyspark.tests.test_serializers
Finished test(pypy3): pyspark.tests.test_serializers (10s)
Starting test(pypy3): pyspark.tests.test_shuffle
Finished test(pypy3): pyspark.tests.test_shuffle (21s)
Starting test(pypy3): pyspark.tests.test_taskcontext
Finished test(pypy3): pyspark.tests.test_rdd (124s)
Starting test(pypy3): pyspark.tests.test_util
Finished test(pypy3): pyspark.tests.test_util (5s)
Starting test(pypy3): pyspark.tests.test_worker
Finished test(pypy3): pyspark.tests.test_worker (21s)
Starting test(python3.6): pyspark.ml.tests.test_algorithms
Finished test(pypy3): pyspark.tests.test_taskcontext (88s)
Starting test(python3.6): pyspark.ml.tests.test_base
Finished test(python3.6): pyspark.ml.tests.test_base (21s)
Starting test(python3.6): pyspark.ml.tests.test_evaluation
Finished test(python3.6): pyspark.ml.tests.test_evaluation (24s)
Starting test(python3.6): pyspark.ml.tests.test_feature
Finished test(python3.6): pyspark.ml.tests.test_algorithms (115s)
Starting test(python3.6): pyspark.ml.tests.test_image
Finished test(python3.6): pyspark.ml.tests.test_feature (45s)
Starting test(python3.6): pyspark.ml.tests.test_linalg
Finished test(python3.6): pyspark.ml.tests.test_image (18s)
Starting test(python3.6): pyspark.ml.tests.test_param
Finished test(python3.6): pyspark.ml.tests.test_param (23s)
Starting test(python3.6): pyspark.ml.tests.test_persistence
Finished test(python3.6): pyspark.ml.tests.test_linalg (40s)
Starting test(python3.6): pyspark.ml.tests.test_pipeline
Finished test(python3.6): pyspark.ml.tests.test_pipeline (7s)
Starting test(python3.6): pyspark.ml.tests.test_stat
Finished test(python3.6): pyspark.ml.tests.test_stat (23s)
Starting test(python3.6): pyspark.ml.tests.test_training_summary
Finished test(python3.6): pyspark.ml.tests.test_persistence (95s)
Starting test(python3.6): pyspark.ml.tests.test_tuning
Finished test(python3.6): pyspark.ml.tests.test_training_summary (59s)
Starting test(python3.6): pyspark.ml.tests.test_util
Finished test(python3.6): pyspark.ml.tests.test_util (36s)
Starting test(python3.6): pyspark.ml.tests.test_wrapper
Finished test(python3.6): pyspark.ml.tests.test_wrapper (29s)
Starting test(python3.6): pyspark.streaming.tests.test_context
Finished test(python3.6): pyspark.streaming.tests.test_context (31s)
Starting test(python3.6): pyspark.streaming.tests.test_dstream
Finished test(python3.6): pyspark.streaming.tests.test_dstream (124s)
Starting test(python3.6): pyspark.streaming.tests.test_kinesis
Finished test(python3.6): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(python3.6): pyspark.streaming.tests.test_listener
Finished test(python3.6): pyspark.streaming.tests.test_listener (14s)
Starting test(python3.6): pyspark.tests.test_appsubmit
Finished test(python3.6): pyspark.tests.test_appsubmit (122s)
Starting test(python3.6): pyspark.tests.test_broadcast
Finished test(python3.6): pyspark.tests.test_broadcast (54s)
Starting test(python3.6): pyspark.tests.test_conf
Finished test(python3.6): pyspark.ml.tests.test_tuning (422s)
Starting test(python3.6): pyspark.tests.test_context
Finished test(python3.6): pyspark.tests.test_conf (16s)
Starting test(python3.6): pyspark.tests.test_daemon
Finished test(python3.6): pyspark.tests.test_daemon (5s)
Starting test(python3.6): pyspark.tests.test_install_spark
Finished test(python3.6): pyspark.tests.test_install_spark (38s)
Starting test(python3.6): pyspark.tests.test_join
Finished test(python3.6): pyspark.tests.test_join (10s)
Starting test(python3.6): pyspark.tests.test_pin_thread
Finished test(python3.6): pyspark.tests.test_context (65s)
Starting test(python3.6): pyspark.tests.test_profiler
Finished test(python3.6): pyspark.tests.test_profiler (14s)
Starting test(python3.6): pyspark.tests.test_rdd
Finished test(python3.6): pyspark.tests.test_pin_thread (42s)
Starting test(python3.6): pyspark.tests.test_rddbarrier
Finished test(python3.6): pyspark.tests.test_rddbarrier (6s)
Starting test(python3.6): pyspark.tests.test_readwrite
Finished test(python3.6): pyspark.tests.test_readwrite (17s)
Starting test(python3.6): pyspark.tests.test_serializers
Finished test(python3.6): pyspark.tests.test_serializers (12s)
Starting test(python3.6): pyspark.tests.test_shuffle
Finished test(python3.6): pyspark.tests.test_shuffle (12s)
Starting test(python3.6): pyspark.tests.test_taskcontext
Finished test(python3.6): pyspark.tests.test_rdd (96s)
Starting test(python3.6): pyspark.tests.test_util
Finished test(python3.6): pyspark.tests.test_util (5s)
Starting test(python3.6): pyspark.tests.test_worker
Finished test(python3.6): pyspark.tests.test_worker (20s)
Starting test(python3.9): pyspark.ml.tests.test_algorithms
Finished test(python3.6): pyspark.tests.test_taskcontext (64s)
Starting test(python3.9): pyspark.ml.tests.test_base
Finished test(python3.9): pyspark.ml.tests.test_base (20s)
Starting test(python3.9): pyspark.ml.tests.test_evaluation
Finished test(python3.9): pyspark.ml.tests.test_evaluation (25s)
Starting test(python3.9): pyspark.ml.tests.test_feature
Finished test(python3.9): pyspark.ml.tests.test_feature (43s)
Starting test(python3.9): pyspark.ml.tests.test_image
Finished test(python3.9): pyspark.ml.tests.test_algorithms (112s)
Starting test(python3.9): pyspark.ml.tests.test_linalg
Finished test(python3.9): pyspark.ml.tests.test_image (18s)
Starting test(python3.9): pyspark.ml.tests.test_param
Finished test(python3.9): pyspark.ml.tests.test_param (22s)
Starting test(python3.9): pyspark.ml.tests.test_persistence
Finished test(python3.9): pyspark.ml.tests.test_linalg (39s)
Starting test(python3.9): pyspark.ml.tests.test_pipeline
Finished test(python3.9): pyspark.ml.tests.test_pipeline (7s)
Starting test(python3.9): pyspark.ml.tests.test_stat
Finished test(python3.9): pyspark.ml.tests.test_stat (20s)
Starting test(python3.9): pyspark.ml.tests.test_training_summary
Finished test(python3.9): pyspark.ml.tests.test_training_summary (55s)
Starting test(python3.9): pyspark.ml.tests.test_tuning
Finished test(python3.9): pyspark.ml.tests.test_persistence (89s)
Starting test(python3.9): pyspark.ml.tests.test_util
Finished test(python3.9): pyspark.ml.tests.test_util (33s)
Starting test(python3.9): pyspark.ml.tests.test_wrapper
Finished test(python3.9): pyspark.ml.tests.test_wrapper (25s)
Starting test(python3.9): pyspark.streaming.tests.test_context
Finished test(python3.9): pyspark.streaming.tests.test_context (28s)
Starting test(python3.9): pyspark.streaming.tests.test_dstream
Finished test(python3.9): pyspark.streaming.tests.test_dstream (120s)
Starting test(python3.9): pyspark.streaming.tests.test_kinesis
Finished test(python3.9): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(python3.9): pyspark.streaming.tests.test_listener
Finished test(python3.9): pyspark.streaming.tests.test_listener (12s)
Starting test(python3.9): pyspark.tests.test_appsubmit
Finished test(python3.9): pyspark.tests.test_appsubmit (109s)
Starting test(python3.9): pyspark.tests.test_broadcast
Finished test(python3.9): pyspark.ml.tests.test_tuning (376s)
Starting test(python3.9): pyspark.tests.test_conf
Finished test(python3.9): pyspark.tests.test_broadcast (48s)
Starting test(python3.9): pyspark.tests.test_context
Finished test(python3.9): pyspark.tests.test_conf (21s)
Starting test(python3.9): pyspark.tests.test_daemon
Finished test(python3.9): pyspark.tests.test_daemon (4s)
Starting test(python3.9): pyspark.tests.test_install_spark
Finished test(python3.9): pyspark.tests.test_install_spark (21s)
Starting test(python3.9): pyspark.tests.test_join
Finished test(python3.9): pyspark.tests.test_join (20s)
Starting test(python3.9): pyspark.tests.test_pin_thread
Finished test(python3.9): pyspark.tests.test_context (71s)
Starting test(python3.9): pyspark.tests.test_profiler
Finished test(python3.9): pyspark.tests.test_profiler (10s)
Starting test(python3.9): pyspark.tests.test_rdd
Finished test(python3.9): pyspark.tests.test_pin_thread (39s)
Starting test(python3.9): pyspark.tests.test_rddbarrier
Finished test(python3.9): pyspark.tests.test_rddbarrier (9s)
Starting test(python3.9): pyspark.tests.test_readwrite
Finished test(python3.9): pyspark.tests.test_readwrite (16s)
Starting test(python3.9): pyspark.tests.test_serializers
Finished test(python3.9): pyspark.tests.test_serializers (8s)
Starting test(python3.9): pyspark.tests.test_shuffle
Finished test(python3.9): pyspark.tests.test_shuffle (13s)
Starting test(python3.9): pyspark.tests.test_taskcontext
Finished test(python3.9): pyspark.tests.test_rdd (95s)
Starting test(python3.9): pyspark.tests.test_util
Finished test(python3.9): pyspark.tests.test_util (4s)
Starting test(python3.9): pyspark.tests.test_worker
Finished test(python3.9): pyspark.tests.test_worker (19s)
Starting test(pypy3): pyspark.accumulators
Finished test(pypy3): pyspark.accumulators (12s)
Starting test(pypy3): pyspark.broadcast
Finished test(python3.9): pyspark.tests.test_taskcontext (70s)
Starting test(pypy3): pyspark.conf
Finished test(pypy3): pyspark.broadcast (7s)
Starting test(pypy3): pyspark.context
Finished test(pypy3): pyspark.conf (7s)
Starting test(pypy3): pyspark.profiler
Finished test(pypy3): pyspark.profiler (13s)
Starting test(pypy3): pyspark.rdd
Finished test(pypy3): pyspark.context (30s)
Starting test(pypy3): pyspark.serializers
Finished test(pypy3): pyspark.serializers (25s)
Starting test(pypy3): pyspark.shuffle
Finished test(pypy3): pyspark.shuffle (3s)
Starting test(pypy3): pyspark.streaming.util
Finished test(pypy3): pyspark.streaming.util (1s)
Starting test(pypy3): pyspark.util
Finished test(pypy3): pyspark.util (1s)
Starting test(python3.6): pyspark.accumulators
Finished test(pypy3): pyspark.rdd (52s)
Starting test(python3.6): pyspark.broadcast
Finished test(python3.6): pyspark.accumulators (14s)
Starting test(python3.6): pyspark.conf
Finished test(python3.6): pyspark.broadcast (9s)
Starting test(python3.6): pyspark.context
Finished test(python3.6): pyspark.conf (6s)
Starting test(python3.6): pyspark.ml.classification
Finished test(python3.6): pyspark.context (29s)
Starting test(python3.6): pyspark.ml.clustering
Finished test(python3.6): pyspark.ml.clustering (73s)
Starting test(python3.6): pyspark.ml.evaluation
Finished test(python3.6): pyspark.ml.classification (119s)
Starting test(python3.6): pyspark.ml.feature
Finished test(python3.6): pyspark.ml.evaluation (33s)
Starting test(python3.6): pyspark.ml.fpm
Finished test(python3.6): pyspark.ml.fpm (36s)
Starting test(python3.6): pyspark.ml.functions
Finished test(python3.6): pyspark.ml.functions (24s)
Starting test(python3.6): pyspark.ml.image
Finished test(python3.6): pyspark.ml.image (9s)
Starting test(python3.6): pyspark.ml.linalg.__init__
Finished test(python3.6): pyspark.ml.linalg.__init__ (0s)
Starting test(python3.6): pyspark.ml.recommendation
Finished test(python3.6): pyspark.ml.feature (107s)
Starting test(python3.6): pyspark.ml.regression
Finished test(python3.6): pyspark.ml.recommendation (53s)
Starting test(python3.6): pyspark.ml.stat
Finished test(python3.6): pyspark.ml.stat (37s)
Starting test(python3.6): pyspark.ml.tuning
Finished test(python3.6): pyspark.ml.regression (73s)
Starting test(python3.6): pyspark.profiler
Finished test(python3.6): pyspark.profiler (11s)
Starting test(python3.6): pyspark.rdd
Finished test(python3.6): pyspark.ml.tuning (52s)
Starting test(python3.6): pyspark.serializers
Finished test(python3.6): pyspark.rdd (48s)
Starting test(python3.6): pyspark.shuffle
Finished test(python3.6): pyspark.shuffle (0s)
Starting test(python3.6): pyspark.streaming.util
Finished test(python3.6): pyspark.streaming.util (0s)
Starting test(python3.6): pyspark.util
Finished test(python3.6): pyspark.util (0s)
Starting test(python3.9): pyspark.accumulators
Finished test(python3.6): pyspark.serializers (13s)
Starting test(python3.9): pyspark.broadcast
Finished test(python3.9): pyspark.accumulators (10s)
Starting test(python3.9): pyspark.conf
Finished test(python3.9): pyspark.broadcast (9s)
Starting test(python3.9): pyspark.context
Finished test(python3.9): pyspark.conf (6s)
Starting test(python3.9): pyspark.ml.classification
Finished test(python3.9): pyspark.context (28s)
Starting test(python3.9): pyspark.ml.clustering
Finished test(python3.9): pyspark.ml.clustering (68s)
Starting test(python3.9): pyspark.ml.evaluation
Finished test(python3.9): pyspark.ml.classification (116s)
Starting test(python3.9): pyspark.ml.feature
Finished test(python3.9): pyspark.ml.evaluation (33s)
Starting test(python3.9): pyspark.ml.fpm
Finished test(python3.9): pyspark.ml.fpm (34s)
Starting test(python3.9): pyspark.ml.functions
Finished test(python3.9): pyspark.ml.functions (24s)
Starting test(python3.9): pyspark.ml.image
Finished test(python3.9): pyspark.ml.image (9s)
Starting test(python3.9): pyspark.ml.linalg.__init__
Finished test(python3.9): pyspark.ml.linalg.__init__ (0s)
Starting test(python3.9): pyspark.ml.recommendation
Finished test(python3.9): pyspark.ml.feature (114s)
Starting test(python3.9): pyspark.ml.regression
Finished test(python3.9): pyspark.ml.recommendation (51s)
Starting test(python3.9): pyspark.ml.stat
Finished test(python3.9): pyspark.ml.stat (40s)
Starting test(python3.9): pyspark.ml.tuning
Finished test(python3.9): pyspark.ml.regression (72s)
Starting test(python3.9): pyspark.profiler
Finished test(python3.9): pyspark.profiler (13s)
Starting test(python3.9): pyspark.rdd
Finished test(python3.9): pyspark.ml.tuning (51s)
Starting test(python3.9): pyspark.serializers
Finished test(python3.9): pyspark.serializers (14s)
Starting test(python3.9): pyspark.shuffle
Finished test(python3.9): pyspark.shuffle (0s)
Starting test(python3.9): pyspark.streaming.util
Finished test(python3.9): pyspark.streaming.util (0s)
Starting test(python3.9): pyspark.util
Finished test(python3.9): pyspark.util (8s)
Finished test(python3.9): pyspark.rdd (45s)
Tests passed in 2908 seconds

========================================================================
Running PySpark packaging tests
========================================================================
Constructing virtual env for testing
Using conda virtual environments
Testing pip installation with python 3.6
Using /tmp/tmp.MMTZT5H3SO for virtualenv
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done