Skip to content

Meeting with Tayyab

Original OneNote page: Survey and research - Meeting with Tayyab

Route: meeting/person note

Categories: people_meetings

Source

  • Word export: C:\Users\adeel\OneDrive\100_Knowledge\203_TextCAD\01_Product_Project_Management\TextCAD_Wiki\inbox\onenoteexport\01_PROJECTS\Gauss Compute\Survey and research - Meeting with Tayyab.docx
  • MHT export: C:\Users\adeel\OneDrive\100_Knowledge\203_TextCAD\01_Product_Project_Management\TextCAD_Wiki\inbox\onenoteexport\01_PROJECTS\Gauss Compute\Survey and research - Meeting with Tayyab.mht

Extracted Notes

These notes are extracted from the Word export so the source is visible in the wiki without moving or deleting the original file.

  • Meeting with Tayyab
  • For part level 1000-2000 drawings
  • Restricted to part level drawings
  • They have around 2200 drawings
  • For better results, get more data
  • Natural of drawings matters
  • Multi agent systems - drawing + step - develop a multi agent system for tool selection
  • First started with 400 drawings
  • Notes
  • Do they template match? Raw vision matching
  • Template matching can also be done depending upon the problem
  • 9 labels
  • Extending bounding boxes
  • If giving a small bounding box its easier, than instead of the whole drawing
  • Assembly level drawings - more than one yolo model
  • One model detects the key views
  • Some views are merging with each other
  • At this stage, still exploring assembly level drawings with merging views
  • Local SME drawings are very complex
  • Automotive or ship building
  • If we get some views, 10 views, based on prompt enginering give to the vision language model, cropped bounding box
  • Generic knowledge of vlm to tell
  • Asking vlm to check if this info is complete of incomplete
  • If the info is being cropped
  • Don’t need to fine tune a vlm (gpt 4o)
  • First model will get 10 views
  • Then input it to vlm, then ask the vlm if the bounding boxes are cropped, unique view (a bit confusing)
  • No need to use the opv model (because views are already along x-y axes)
  • For bounding boxed, inclindation yolo v model
  • For paper analysis
  • If the size of checkpoing is higher, thenn inference speed lower
  • Yolo v11m,
  • Industry more interested in x version, has more accuracy
  • For vision language model
  • Florence, open source (general purpose model)
  • Donut (ocr free model)
  • Donut is better, only for information extraction (from the drawing for us)
  • Check latest model at huggingface
  • Since working in a government institute, they approach the university for this support
  • DATA
  • ASME textBooks
  • FOR CAD Models
  • Method
  • GNN based (developed model to extract key features, machining features)
  • For complex like shaft, not suited
  • Python OCC, algorithm to develop geomterical information of
  • Not good, but for higher level
  • Convert complex parts to snapshots
  • Ask VLM what features do you see
  • Most of the papers, detection of features
  • Industrial problem to extract dimensions as well, raidus or orientation of the hole
  • Use mathematical based scripts to extract geometric features
  • Big misconception
  • AI cant do wont 100% job
  • Llm and vlm skip some information
  • He has developed a multi-agent system
  • Applied human in the loop
  • Review some annotations and correct it
  • Manufacturing is precision driven industry, not completely rely on AI
  • Hyundai
  • Machining as a case study
  • More mature system, 100-200 years
  • Working in A-start and NTU
  • May be we can collaborate on NTU side
  • Win-win situation could be a collaboration
  • If not on university level, we can also do consultation
  • Professor: Moon

Next Curation Action

Promote this into a person page, link it to the relevant company/account, and create/update the EspoCRM contact if this relationship is still active.

Sources

  • C:\Users\adeel\OneDrive\100_Knowledge\203_TextCAD\01_Product_Project_Management\TextCAD_Wiki\inbox\onenoteexport\01_PROJECTS\Gauss Compute\Survey and research - Meeting with Tayyab.docx
  • C:\Users\adeel\OneDrive\100_Knowledge\203_TextCAD\01_Product_Project_Management\TextCAD_Wiki\inbox\onenoteexport\01_PROJECTS\Gauss Compute\Survey and research - Meeting with Tayyab.mht