|
|
View Forum Thread
Anyone can view the forums, but you need to log in in order to post messages.
> Forum Home >
Feature Requests >
Smedge Engine able to kill sub-thread immediately upon seeing a string from the task's std.out text/output
|
Fri, 11/Mar/2011 7:53 AM |
Jamie
12 Posts
|
One thing the generic module engine could do with is the ability to immediately kill a thread active on a node once the engine recognises a key trigger word.
I'll explain.
IFD generation for Houdini (using hbatch) being executed on our farm here suffers from the same problematic horrendous slow unloading/deleting assets from a scene syndrome artists have at their desktop which occurs 'after' the IFD has completed writing successfully.
The hbatch process basically cycles through every single asset loaded (unless we're using delayed loads which isn't always practical to use on every shot) a whole raft of completely useless onload/unload/ondelete post render script execution checks.
Now we never implement (en-masse) anything in Hbatch that uses this feature, we've spoke to SideFx and will re-reopening the support line on this again, but is it possible to make smedge 'kill' the thread upon seeing a particular string.
Quite often it's possible to manually permanently cancel the process when you are watching the process output and save up to 10mins per packet of render time, and we have tried using Python wrappers to terminate the child process but it also kills the smedge engine itself.
I may re-visit this with a view to testing kill process python scripts on Smedge 2010 to see if this has changed much since 30242, I think when I was killing thread using name/Proc ID Python wasn't too gentle about it..
But this would streamline IFD generation on the farm here no end and we'd have less chances of bottle-necking up during crunch times etc.
Let me know what you (or anyone else using Houdini with Smedge) thinks.
Best regards
Jamie
Axis
Edited by author on Fri, 11/Mar/2011 7:55 AM |
|
|
|
Fri, 11/Mar/2011 1:00 PM |
Robin
1138 Posts
|
You can do this by adding the string you want to check for to the Error Start strings list. Basically, you are telling Smedge to look for a particular message to consider an "error", and then when that error message is found, Smedge immediately aborts the current work.
All products that are based on "ProcessJob" have this parameter, which is just about every product in Smedge. This would include both the Generic Script module and the ProcessSequence module, which provides the support for the Virtual Modules, including Houdini.
You can set the error start string job by job in the Advanced Info tab, or using the -ErrorStarts flag from the Submit command line. If you want to, you can also set error start strings as an Engine Product Option.
Let me know if that works for you. Thanks,
-robin
|
|
|
|
Tue, 22/Mar/2011 7:49 AM |
Jamie
12 Posts
|
Ah, now as helpful as it is, it's not an error I'm trying to trap, it's a detected completed process signal I'm wanting to fire that kills the sub-thread on that node cleanly.
What you explained is perfect for what we need but it's not an errored state that we're watching for, we'd want the task to be marked/flagged as complete once the node has spotted this '* Render finished *' signal our IFD generation process spits out in the std output.
|
|
|
|
Tue, 22/Mar/2011 2:08 PM |
Robin
1138 Posts
|
Ah, I see. It's basically the same as the error detection, except that it marks the work as successful as soon as the message is received, then stops the work from doing anything further. Is that right?
If so, I'll take a look at adding this kind of "success message" test into Smedge somehow, but it won't make it into the 2011 release that just came out.
Thanks
-robin
|
|
|
|
Wed, 23/Mar/2011 11:26 AM |
Jamie
12 Posts
|
That's quite ok Robin, I'll give you an update if SideFX act on a feature request to also sort this out. I posted it as a bug to them years ago, they came back and said it was a feature request really since that's just how Houdini works, despite hours of wasted farm time on the IFD generation front.
cheers
Jamie
Axis
|
|
|
|
|
|
. |