KeyError: 'mode': utils.py", line 1126, in parse_metrics if data["mode"] == "validation": #200

New Issue

chigkim · 2023-04-12T12:41:30Z

chigkim commented

2023-04-12 12:41:30 +00:00

I'm trying to resume training, but I get this error.

[Training] [2023-04-12T12:34:58.211878] 23-04-12 12:34:58.211 - INFO: Resuming training from epoch: 37, iter: 75.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/gradio/routes.py", line 395, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 1193, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 930, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.9/dist-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.9/dist-packages/gradio/utils.py", line 491, in async_iteration
    return next(iterator)
  File "/content/ai-voice-cloning/src/utils.py", line 1439, in run_training
    result, percent, message = training_state.parse( line=line, verbose=verbose, keep_x_past_checkpoints=keep_x_past_checkpoints, progress=progress )
  File "/content/ai-voice-cloning/src/utils.py", line 1362, in parse
    self.load_statistics()
  File "/content/ai-voice-cloning/src/utils.py", line 1303, in load_statistics
    self.parse_metrics(data)
  File "/content/ai-voice-cloning/src/utils.py", line 1126, in parse_metrics
    if data["mode"] == "validation":
KeyError: 'mode'

I'm trying to resume training, but I get this error. ``` [Training] [2023-04-12T12:34:58.211878] 23-04-12 12:34:58.211 - INFO: Resuming training from epoch: 37, iter: 75. Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/gradio/routes.py", line 395, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 1193, in process_api result = await self.call_function( File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 930, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.9/dist-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/usr/local/lib/python3.9/dist-packages/gradio/utils.py", line 491, in async_iteration return next(iterator) File "/content/ai-voice-cloning/src/utils.py", line 1439, in run_training result, percent, message = training_state.parse( line=line, verbose=verbose, keep_x_past_checkpoints=keep_x_past_checkpoints, progress=progress ) File "/content/ai-voice-cloning/src/utils.py", line 1362, in parse self.load_statistics() File "/content/ai-voice-cloning/src/utils.py", line 1303, in load_statistics self.parse_metrics(data) File "/content/ai-voice-cloning/src/utils.py", line 1126, in parse_metrics if data["mode"] == "validation": KeyError: 'mode' ```

mrq commented

2023-04-12 20:04:33 +00:00

Should be fixed in the latest commit.