Skip to content

Commit

Permalink
Removing deprecated pyarrow.hdfs.connect call from iorw.py (#615)
Browse files Browse the repository at this point in the history
* SInce v2.0: pyarrow.hdfs.connect is deprecated, calling pyarrow.fs.HadoopFileSystem instead.

Addressing commonly see warning in iorw.py file.

/usr/local/lib/python3.7/dist-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.
  from pyarrow import HadoopFileSystem

* Update papermill/iorw.py

Co-authored-by: Matthew Seal <mseal007@gmail.com>
  • Loading branch information
abhimanyu-bitsgoa and MSeal committed Jun 16, 2021
1 parent d30bcf2 commit e2f23ca
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion papermill/iorw.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,11 @@
except ImportError:
GCSFileSystem = missing_dependency_generator("gcsfs", "gcs")
try:
from pyarrow import HadoopFileSystem
try:
from pyarrow.fs import HadoopFileSystem
except ImportError:
# Attempt the older package import pattern in case we're using an old dep version.
from pyarrow import HadoopFileSystem
except ImportError:
HadoopFileSystem = missing_dependency_generator("pyarrow", "hdfs")

Expand Down

0 comments on commit e2f23ca

Please sign in to comment.