Having hardtime making Whispercpp and Whisperx work (COLAB) #195

New Issue

pheonis · 2023-04-08T19:01:01Z

pheonis commented

2023-04-08 19:01:01 +00:00

I had created an issue a few days back on this topic, that time i was getting "No module named whispercpp or whisperx".

Now i have installed Whisper x and whispercpp. Using the following methods respectively.

!git clone https://git.ecker.tech/lightmare/whispercpp.py
%cd whispercpp.py
!git submodule update --init
!pip install .

%cd /content/ai-voice-cloning/
!git clone https://github.com/m-bain/whisperX.git
%cd whisperX
!pip install -e .

Upto this step , everything was good ,until i tried using them. And these are the errors im getting respectively.

When i try to use whispercpp.i can see the models get downloaded and loaded but after that the instance hangs and i get "connection errored out" message.

Upon reloading, i get 504 Gateway Time-out error on gradio link and gradio instance stops and i have to restart the start.sh script all over again.

Loading Whisper model: small.en
whisper_init_from_file: loading model from 'models/ggml-small.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head  = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 768
whisper_model_load: n_text_head   = 12
whisper_model_load: n_text_layer  = 12
whisper_model_load: n_mels        = 80
whisper_model_load: f16           = 1
whisper_model_load: type          = 3
whisper_model_load: mem required  =  603.00 MB (+   16.00 MB per decoder)
whisper_model_load: kv self size  =   15.75 MB
whisper_model_load: kv cross size =   52.73 MB
whisper_model_load: adding 1607 extra tokens
whisper_model_load: model ctx     =  464.56 MB
whisper_model_load: model size    =  464.44 MB
Loaded Whisper model

While using whisperx, Im getting this error on the webui

/usr/local/lib/python3.9/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops10select_int4callERKNS_6TensorElN3c106SymIntE

This is the output in colab:

OSError Traceback (most recent call last)
<ipython-input-43-a5d22c61363b> in <cell line: 1>()
----> 1 from pyannote.audio import Pipeline
2 import os
3 import subprocess
4 from pydub import AudioSegment
5 from pydub.silence import split_on_silence
15 frames
/usr/lib/python3.9/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error, winmode)
372
373 if handle is None:
--> 374 self._handle = _dlopen(self._name, mode)
375 else:
376 self._handle = handle
OSError: /usr/local/lib/python3.9/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops10select_int4callERKNS_6TensorElN3c106SymIntE

What am i doing wrong?

I had created an issue a few days back on this topic, that time i was getting "No module named whispercpp or whisperx". Now i have installed Whisper x and whispercpp. Using the following methods respectively. ``` !git clone https://git.ecker.tech/lightmare/whispercpp.py %cd whispercpp.py !git submodule update --init !pip install . ``` ``` %cd /content/ai-voice-cloning/ !git clone https://github.com/m-bain/whisperX.git %cd whisperX !pip install -e . ``` Upto this step , everything was good ,until i tried using them. And these are the errors im getting respectively. When i try to use whispercpp.i can see the models get downloaded and loaded but after that the instance hangs and i get "connection errored out" message. Upon reloading, i get 504 Gateway Time-out error on gradio link and gradio instance stops and i have to restart the start.sh script all over again. ``` Loading Whisper model: small.en whisper_init_from_file: loading model from 'models/ggml-small.en.bin' whisper_model_load: loading model whisper_model_load: n_vocab = 51864 whisper_model_load: n_audio_ctx = 1500 whisper_model_load: n_audio_state = 768 whisper_model_load: n_audio_head = 12 whisper_model_load: n_audio_layer = 12 whisper_model_load: n_text_ctx = 448 whisper_model_load: n_text_state = 768 whisper_model_load: n_text_head = 12 whisper_model_load: n_text_layer = 12 whisper_model_load: n_mels = 80 whisper_model_load: f16 = 1 whisper_model_load: type = 3 whisper_model_load: mem required = 603.00 MB (+ 16.00 MB per decoder) whisper_model_load: kv self size = 15.75 MB whisper_model_load: kv cross size = 52.73 MB whisper_model_load: adding 1607 extra tokens whisper_model_load: model ctx = 464.56 MB whisper_model_load: model size = 464.44 MB Loaded Whisper model ``` While using whisperx, Im getting this error on the webui ``` /usr/local/lib/python3.9/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops10select_int4callERKNS_6TensorElN3c106SymIntE ``` This is the output in colab: ``` OSError Traceback (most recent call last) <ipython-input-43-a5d22c61363b> in <cell line: 1>() ----> 1 from pyannote.audio import Pipeline 2 import os 3 import subprocess 4 from pydub import AudioSegment 5 from pydub.silence import split_on_silence 15 frames /usr/lib/python3.9/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error, winmode) 372 373 if handle is None: --> 374 self._handle = _dlopen(self._name, mode) 375 else: 376 self._handle = handle OSError: /usr/local/lib/python3.9/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops10select_int4callERKNS_6TensorElN3c106SymIntE ``` What am i doing wrong?

psammites commented

2023-04-08 21:05:42 +00:00

I wouldn't clone them in a subdirectory of ai-voice-cloning just in case the different venv's conflict.

pheonis commented

2023-04-09 00:51:33 +00:00

I wouldn't clone them in a subdirectory of ai-voice-cloning just in case the different venv's conflict.

so, how would you install? right in the main ai-voice-cloning directory?

> I wouldn't clone them in a subdirectory of ai-voice-cloning just in case the different venv's conflict. so, how would you install? right in the main ai-voice-cloning directory?

pheonis commented

2023-04-09 02:27:21 +00:00

the different venv's conflict.

I think the venv is not getting activated when executing setup_cuda.sh or start.sh.

"./venv/bin/activate: No such file or directory"
I can check in the ./venv/bin/ directory and there is not activate script to open.

And This is the error i get everytime, although colab notebook is running fine, i was able to generate voice clips and train models. but i think because this virtualenvironment is not activating at first,thats why im getting all the errors when playing with different whispers.

>the different venv's conflict. I think the venv is not getting activated when executing setup_cuda.sh or start.sh. `"./venv/bin/activate: No such file or directory"` I can check in the ./venv/bin/ directory and there is not activate script to open. And This is the error i get everytime, although colab notebook is running fine, i was able to generate voice clips and train models. but i think because this virtualenvironment is not activating at first,thats why im getting all the errors when playing with different whispers.

psammites commented

2023-04-09 03:31:04 +00:00

so, how would you install? right in the main ai-voice-cloning directory?

I do pip install git+https://github.com/m-bain/whisperx.git in my home directory, but keep in mind I've never used COLAB so I don't know if that's something you can do there.

> so, how would you install? right in the main ai-voice-cloning directory? I do `pip install git+https://github.com/m-bain/whisperx.git` in my home directory, but keep in mind I've never used COLAB so I don't know if that's something you can do there.

pheonis commented

2023-04-09 05:21:04 +00:00

so, how would you install? right in the main ai-voice-cloning directory?

I do pip install git+https://github.com/m-bain/whisperx.git in my home directory, but keep in mind I've never used COLAB so I don't know if that's something you can do there.

I did install like that too,but didnt workout. Now as far as i understood, google colab doesnt activate the venv virtual environment using the "!source ./venv/bin/activate"..so everything gets installed outside the virtual environment and conflicts of dependencies happening and ultimately the whisperx/whispercpp not working.

> > so, how would you install? right in the main ai-voice-cloning directory? > > I do `pip install git+https://github.com/m-bain/whisperx.git` in my home directory, but keep in mind I've never used COLAB so I don't know if that's something you can do there. I did install like that too,but didnt workout. Now as far as i understood, google colab doesnt activate the venv virtual environment using the "!source ./venv/bin/activate"..so everything gets installed outside the virtual environment and conflicts of dependencies happening and ultimately the whisperx/whispercpp not working.

Sign in to join this conversation.