Welcome, Guest
Username: Password: Remember me

TOPIC: cannot read/load results.slf run_telfile.py scan or res=TelemacFile()

cannot read/load results.slf run_telfile.py scan or res=TelemacFile() 3 months 1 week ago #44103

  • phmusiedlak
  • phmusiedlak's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 7
  • Thank you received: 1
Hello everyone,

I made a one year T2D simulation of the whole English channel: results.slf is pretty big 7Go. I cannot load it using the usual Python library TelemacFile from
data_manip.extraction.telemac_file
.

I also tried to run :
run_tefile.py scan --data results.slf
or
extract.py timeseries results.slf aa.csv --v "VELOCITY U" --node 6 66 666

All ended up with the same error:
struct.error: unpack requires a buffer of 67964 bytes

my guess is that it requires way too much RAM to be loaded.

However, I have no problem opening the case with BlueKenue and extracting the timeseries at my points of interest (as long as I am patient enough).

The solution I am aiming at is to re-run my case while splitting the it in 2 months run.

Any other solution? Is this a known limit of the extract library ?

Just in case, I have uploaded the .slf here (available for a month): www.swisstransfer.com/d/e6722fff-bc9a-4074-9af7-e97dcca49064

Thanks,

Pierre-Henri
The administrator has disabled public write access.

cannot read/load results.slf run_telfile.py scan or res=TelemacFile() 3 months 1 week ago #44104

  • c.coulet
  • c.coulet's Avatar
  • OFFLINE
  • Moderator
  • Posts: 3632
  • Thank you received: 1010
Hi

Not gone to this result size but I had no problem up to 4 Go...
By the way, few remarks

Supposing you run in parallel, why not using the splitted results. It needs few extra tests to identify the subdomain but result file will be lighter
Did you run your python on your PC or a cluster?
Windows or Linux?

My personal experience leads me to manage as much as possible my large result file on our cluster, under Linux...

Regards
Christophe
The administrator has disabled public write access.

cannot read/load results.slf run_telfile.py scan or res=TelemacFile() 3 months 6 days ago #44116

  • phmusiedlak
  • phmusiedlak's Avatar
  • OFFLINE
  • Fresh Boarder
  • Posts: 7
  • Thank you received: 1
Hello,

I did run in parallel using 4 procs. Telemac and post-processing both run on my linux windows sub-system (12 procs available and 30Go of RAM). I guess Python runs in serial.

I did not do anything specific for the parallel , nor the splitting or the merging as I simply do :
telemac2d.py tel.cas --ncsize=4

And I do not know how to just load/use the splitted slf made during the parallel. I will have a look in the documentations.

Thanks

Pierre-Henri
The administrator has disabled public write access.
Moderators: borisb

The open TELEMAC-MASCARET template for Joomla!2.5, the HTML 4 version.