I've already installed sparknlp and its assembly jars, but I still get an error when I try to use one of the models, I get a TypeError: 'JavaPackage' object is not callable.

I cannot install the model and load it from disk because it's considered too big (>100MB) to my project, so I've been suggested to use HDFS to load the pretrained model. Is there a way to do that?

My code:

    from sparknlp.pretrained import PretrainedPipeline
    spark = sparknlp.start()
    pipeline = PretrainedPipeline('analyze_sentimentdl_glove_imdb', lang = 'en')
    annotations =  pipeline.fullAnnotate("Hello from John Snow Labs ! ")[0]

What would be the equivalent for loading with HDFS?

EDIT: Full traceback:

Using NLU code:

Traceback (most recent call last):

  File "/usr/local/juicer/juicer/spark/spark_minion.py", line 490, in _perform_execute
    raise ex from None

  File "/usr/local/juicer/juicer/spark/spark_minion.py", line 486, in _perform_execute
    self._emit_event(room=job_id, namespace='/stand'))

  File "/tmp/juicer_app_10_10_60.py", line 230, in main
    task_futures['a6d45e1d-4322-443e-b7e9-ed78b504a8b0'].result()

  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 435, in result
    return self.__get_result()

  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception

  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)

  File "/tmp/juicer_app_10_10_60.py", line 228, in <lambda>
    lambda: sentimentanalysis_1(spark_session, cached_state, cached_emit_event))

  File "/tmp/juicer_app_10_10_60.py", line 150, in sentimentanalysis_1
    result_df = nlu.load('emotion').predict("I am so happy")

  File "/usr/local/lib/python3.7/dist-packages/nlu/__init__.py", line 153, in load
    f"Something went wrong during creating the Spark NLP model for your request =  {request} Did you use a NLU Spell?")

Exception: Something went wrong during creating the Spark NLP model for your request =  emotion Did you use a NLU Spell?

Using my original code (spark nlu):

Traceback (most recent call last):

  File "/usr/local/juicer/juicer/spark/spark_minion.py", line 490, in _perform_execute
    raise ex from None

  File "/usr/local/juicer/juicer/spark/spark_minion.py", line 486, in _perform_execute
    self._emit_event(room=job_id, namespace='/stand'))

  File "/tmp/juicer_app_10_10_51.py", line 225, in main
    task_futures['a6d45e1d-4322-443e-b7e9-ed78b504a8b0'].result()

  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 435, in result
    return self.__get_result()

  File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception

  File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)

  File "/tmp/juicer_app_10_10_51.py", line 223, in <lambda>
    lambda: sentimentanalysis_1(spark_session, cached_state, cached_emit_event))

  File "/tmp/juicer_app_10_10_51.py", line 145, in sentimentanalysis_1
    pipeline = PretrainedPipeline('analyze_sentimentdl_glove_imdb', lang = 'en')

  File "/usr/local/lib/python3.7/dist-packages/sparknlp/pretrained.py", line 141, in __init__
    self.model = ResourceDownloader().downloadPipeline(name, lang, remote_loc)

  File "/usr/local/lib/python3.7/dist-packages/sparknlp/pretrained.py", line 72, in downloadPipeline
    file_size = _internal._GetResourceSize(name, language, remote_loc).apply()

  File "/usr/local/lib/python3.7/dist-packages/sparknlp/internal.py", line 232, in __init__
    "com.johnsnowlabs.nlp.pretrained.PythonResourceDownloader.getDownloadSize", name, language, remote_loc)

  File "/usr/local/lib/python3.7/dist-packages/sparknlp/internal.py", line 165, in __init__
    self._java_obj = self.new_java_obj(java_obj, *args)

  File "/usr/local/lib/python3.7/dist-packages/sparknlp/internal.py", line 175, in new_java_obj
    return self._new_java_obj(java_class, *args)

  File "/usr/local/spark/python/pyspark/ml/wrapper.py", line 67, in _new_java_obj
    return java_obj(*java_args)

TypeError: 'JavaPackage' object is not callabl