Instead of a standard SW benchmark we need...

Discussion in 'SolidWorks' started by Eddy Hicks, Feb 13, 2004.

  1. Eddy Hicks

    Eddy Hicks Guest

    <<dreaming here>>

    We need something that could press SW to its limits the same way every time
    to force a crash if a crash is inevitable. That way, we could easily tell
    if a SP or a new piece of hardware or software added to our system is going
    to be more or less reliable than what we're using at the time. I wonder if
    some SW programming guru could think about writing a public domain "burn in"
    utility for SW. Something that could run a suite of tests, creating solid
    models, assy's, drawings, using sweeps, lofts, extrudes, imports, etc., you
    know all the normal stuff. And when you want to test, you just set it up
    and walk away until it crashes. Then you go read the logs; how long did it
    take, what was it doing, etc.

    If there's ten thousand different things that can make SW crash then I'm
    sure some AI software could cover at least a few thousand of those things.
    If an SP can't make it past 5 hours then it doesn't get used.

    Hmmm, I wonder if it's possible. That's how you do it with burn in software
    when testing new hardware.

    Any thoughts?

    - Eddy
    Eddy Hicks, Feb 13, 2004
  2. Would the model be built from scratch via a macro or

    I remember seeing a post many months ago where someone was
    looking for the ultimate complex model. My thought at the time
    was that the ultimate model would take a month to load ;^)

    I'm not sure what came of it, but now that I think about it,
    maybe you can modularize this by making an assembly of
    many parts. Each part would demonstrate a particular
    feature or technique.

    You can make a list of every feature in SW and assign a
    different volunteer for each. They would send in a pre-
    determined number of examples either in one part (with
    several features) or several parts (one feature each).

    For example...

    Volunteer: Joe User
    Assigned Feature: Surface Sweep

    Joe sends in...
    A helical Sweep
    A Sweep with Guide Curves
    A Sweep demonstrating Start/End tangencies
    A Sweep demonstrating "Keep Normal Constant"
    etc, etc.

    You can have a macro open each part, do a rebuild,
    report any errors, changes to faces, total rebuild times

    This would be a good benchmark to compare individual
    features, but of course in the real world a part is made
    of of many different features.

    Are you thinking of something like the SolidSolutions
    benchmark? It takes an existing assembly and does
    things with it.

    Mike Wilson
    Mike J. Wilson, Feb 13, 2004
  3. Eddy Hicks

    Eddy Hicks Guest

    Yeah, I guess the SolidSolutions (now Spec) benchmark spurred the idea. And
    I like the idea of volunteers being assigned a feature or function. And the
    idea of a code snippet or macro putting it all together. Might be a good
    way to start things off. Maybe a VB applet that loads parts from a home
    folder until it runs out. Maybe something along the lines of...

    "IF something found THEN add to assy and perform actions and log it all ELSE
    stop" - could get this beast underway. And if people are so inclined or
    think they've got a new way to break Solidworks they could volunteer a new
    part or subassy and the applet would pick up on it just by finding it in
    that home folder.

    Here's what I'm thinking now Mike...

    1) an applet is coded that creates unique and specific parts, based on a
    specific feature or set of features (extrude+shell+edits,
    sweep+loft+surface+thicken+edits, etc.). This applet locates these parts in
    a home folder.

    2) Users create parts and subassys of challenging features or things that
    have been known to cause crashes for them in the past. These parts and
    subassys would be added to the home folder.

    3) An applet is created, whose job it is to create a parent assy, adding the
    pieces from 1&2 above - only once for each unique piece, until it's parsed
    the entire folder. This applet performs specific actions on the parent assy
    as well as specific pieces of the assy, like in-context edits to the parts
    created in step 1, along with zooms, pans, rotates, hiding, showing,
    suppressing, lightweighting, resolving, etc. and all these functions are
    logged to a text file. It runs until it SW crashes or until a user stops
    it. But every action is written to the log before it's performed so when
    there's a crash you know what caused it.

    4) As more users come up with more pieces, these pieces get added to the
    home folder and hence, added to the parent assy via the main applet.

    Hmmm, maybe this isn't such a weird idea? :)

    - Eddy
    Eddy Hicks, Feb 13, 2004
  4. It was me, and actually I was looking for the most complex PART in the sense
    of many different features, for a documentation tool. Check Food Tray.SLDPRT.xml for the one I got
    (and send me your part if you have a more complex one)
    I'd say the SW demo models are a good start in that sense.

    A proposal for a benchmark procedure could be:
    1) run EcoSqueeze with the option to remove the parasolid info to "clear the
    2) run a macro with "ForceRebuild(false)" to rebuild everything in the
    3) get the rebuild statistics from SW (will look how from API)
    4) compare the geometry (how ?)

    Philippe Guglielmetti -
    Philippe Guglielmetti, Feb 13, 2004
  5. --
    Just curious, my browser won't let me view this xml file. I downloaded the
    ImageMagick, but still no picture...

    Not Necessarily Me, Feb 13, 2004
  6. Eddy Hicks

    R. Wink Guest

    Tried to follow your reference back to dl the part and assembly. It crashed, saying that the files are not on the server,
    please contact the web master.
    R. Wink
    R. Wink, Feb 13, 2004
  7. Philippe Guglielmetti, Feb 13, 2004
  8. Eddy Hicks

    kenneth b Guest

    You cannot follow the links as the .sldprt model is not publicly available

    i can't believe you've included "huge size" as an excuse/reason. my cable
    internet connection has absolutely zero problems downloading extremely large
    files. :)
    kenneth b, Feb 13, 2004
  9. Eddy Hicks

    matt Guest

    Server space is often limited. I've had to remove stuff from my site to
    make room for other stuff. It sounds like a valid reason to me.

    matt, Feb 13, 2004
  10. Eddy Hicks

    kenneth b Guest

    ooops, didn't think of that
    kenneth b, Feb 13, 2004
  11. Eddy Hicks

    MM Guest


    Sounds allot like what SW "claims" to do already as part of their standard
    testing. When I was there, they had a whole room full of machines (different
    hardware configs) running real time tests. How effective their test suites
    were/are is open to debate. I didn't see exactly "what" functionality they
    were testing, or to what depth.

    Even then, the answers I got led me to believe that they made decisions
    "SOLELY" on statistical/bean counter/marketing data. This probably means
    that they only test for the level of functionality needed by the average
    user. Anything more comprehensive would probably be construed as not "cost
    effective". This was blindingly evident back then.

    They seem to have gotten somewhat better as of late, but it sure wouldn't
    hurt to have an independent test routine. The big question, to me at least,
    is would they bother to verify and honor the results. If so, would they act
    on them.


    MM, Feb 15, 2004
  12. Eddy Hicks

    Eddy Hicks Guest

    Hey Mark,

    My idea is based on us, the users, doing tests to get the software to fail.
    Mostly to justify to ourselves whether a specific SP fixes what was broken
    to us, and whether it broke anything else in the process. I think the whole
    thing is do-able but whether it's a benefit to SW Corp never even entered my
    mind. The idea was to save us testing time, not them. That would only be
    an ancillary benefit.

    - Eddy
    Eddy Hicks, Feb 15, 2004
  13. Eddy Hicks

    MM Guest


    Would save allot manual testing I suppose. If it were flexible enough to be
    customized to individual companies, it would even be worth buying.

    By the way, I bought the MSI "K8T Master2-FAR" with two 246's, 2gb ram, and
    a FX1000. I should get the parts on Monday.


    MM, Feb 16, 2004
  14. Eddy Hicks

    Eddy Hicks Guest

    Awesome! Should be a screamer. Like I said, can't go wrong with MSI. Did
    you remember to buy "registered" ram? :)

    - Eddy
    Eddy Hicks, Feb 16, 2004
  15. Oh yeah, I was going to ask about this. Would this make a
    good product? I would think there would be more interest
    in the project if there were profits to be had. Of course
    since this could turn into a monster, one may not have a
    choice but to sell it?

    Mike Wilson
    Mike J. Wilson, Feb 16, 2004
Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.