Maybe dumb question; Could running multiple instances share the same models/resources? #273

New Issue

wiznat · 2023-06-20T06:06:14Z

wiznat commented

2023-06-20 06:06:14 +00:00

Pretty much what the question says. I'm trying a funky experiment to run two TTTS at a time, but running each one eats up about 5g of vram after loading the diffusion model, vocoder, diffusion model.

I don't know enough about ML to say if them sharing these would be possible, but just figured Id ask here before trying to write some custom setup to share the resources.

Pretty much what the question says. I'm trying a funky experiment to run two TTTS at a time, but running each one eats up about 5g of vram after loading the diffusion model, vocoder, diffusion model. I don't know enough about ML to say if them sharing these would be possible, but just figured Id ask here before trying to write some custom setup to share the resources.

psammites commented

2023-06-20 13:05:21 +00:00

Unless your time is worthless the amount of development (not just coding but testing and debugging) this would require would likely cost more than buying a second GPU.

😆 1

helloitsme commented

2023-06-20 21:23:12 +00:00

I am able to run multiple instances on Google Colab, using the same model stored on Google Drive.

wiznat commented

2023-06-20 22:16:04 +00:00

I am able to run multiple instances on Google Colab, using the same model stored on Google Drive.

I originally was doing this but for my purposes the speed was not adequate and too costly to run multiple colabs for as long as I need, so I just went with an EC2 which can run 2 at a time and get much faster times piping the results to my needed destination.

> I am able to run multiple instances on Google Colab, using the same model stored on Google Drive. I originally was doing this but for my purposes the speed was not adequate and too costly to run multiple colabs for as long as I need, so I just went with an EC2 which can run 2 at a time and get much faster times piping the results to my needed destination.

helloitsme commented

2023-06-21 18:45:17 +00:00

I am able to run multiple instances on Google Colab, using the same model stored on Google Drive.

I originally was doing this but for my purposes the speed was not adequate and too costly to run multiple colabs for as long as I need, so I just went with an EC2 which can run 2 at a time and get much faster times piping the results to my needed destination.

The $/hr is better too? A T4 with google is $0.40/hr.

> > I am able to run multiple instances on Google Colab, using the same model stored on Google Drive. > > I originally was doing this but for my purposes the speed was not adequate and too costly to run multiple colabs for as long as I need, so I just went with an EC2 which can run 2 at a time and get much faster times piping the results to my needed destination. The $/hr is better too? A T4 with google is $0.40/hr.

wiznat commented

2023-06-23 06:15:04 +00:00

I am able to run multiple instances on Google Colab, using the same model stored on Google Drive.

I originally was doing this but for my purposes the speed was not adequate and too costly to run multiple colabs for as long as I need, so I just went with an EC2 which can run 2 at a time and get much faster times piping the results to my needed destination.

The $/hr is better too? A T4 with google is $0.40/hr.

That's good to know actually. I only opted for the EC2 because I have AWS experience and already had the quota needed to run a GPU vm from other projects. I just looked and they also have a 2 T4 option unlike AWS ! Guess I gotta wait for my quota increase from google now lol

> > > I am able to run multiple instances on Google Colab, using the same model stored on Google Drive. > > > > I originally was doing this but for my purposes the speed was not adequate and too costly to run multiple colabs for as long as I need, so I just went with an EC2 which can run 2 at a time and get much faster times piping the results to my needed destination. > > The $/hr is better too? A T4 with google is $0.40/hr. That's good to know actually. I only opted for the EC2 because I have AWS experience and already had the quota needed to run a GPU vm from other projects. I just looked and they also have a 2 T4 option unlike AWS ! Guess I gotta wait for my quota increase from google now lol

helloitsme commented

2023-06-23 06:42:07 +00:00

Actually, I just realized it's $0.20/hr: $10 for 100 compute resources, 2 computes per hour for a T4= 50 hours...$10/50 = $0.2/hr

wiznat commented

2023-06-23 08:57:16 +00:00

Actually, I just realized it's $0.20/hr: $10 for 100 compute resources, 2 computes per hour for a T4= 50 hours...$10/50 = $0.2/hr

Ohh I thought you meant with like a Google cloud VM, which just so happens is actually $.40 and hour lol.

While it would (probably) be cheaper on colab, it came down to a speed requirement.

On colab, the voice gen was taking too slow, short lines were around 7-8 seconds, and longer lines sometimes up to 20 seconds. I'm generating constant 'scenes' of dialogue, so if the scene play time is less than the scene gen time, I get a delay/pause.

With the VM, I can get short lines in about 3-4 seconds, and longer lines no more than 12 seconds. Plus, with colab I had to package and wait for it in google drive, + download it, which ate up a decent amount of time usually 30+ seconds. With the VM I can just SSH directly to my machine, and its practically zero wait time.

> Actually, I just realized it's $0.20/hr: $10 for 100 compute resources, 2 computes per hour for a T4= 50 hours...$10/50 = $0.2/hr Ohh I thought you meant with like a Google cloud VM, which just so happens is actually $.40 and hour lol. While it would (probably) be cheaper on colab, it came down to a speed requirement. On colab, the voice gen was taking too slow, short lines were around 7-8 seconds, and longer lines sometimes up to 20 seconds. I'm generating constant 'scenes' of dialogue, so if the scene play time is less than the scene gen time, I get a delay/pause. With the VM, I can get short lines in about 3-4 seconds, and longer lines no more than 12 seconds. Plus, with colab I had to package and wait for it in google drive, + download it, which ate up a decent amount of time usually 30+ seconds. With the VM I can just SSH directly to my machine, and its practically zero wait time.

Sign in to join this conversation.