[Thread Prev][Thread Next][Index]

[ferret_users] pyferret fails to read a very long directory



Dear all,
 
I am trying to glue a large amount of files using the aggregate capability of ferret. The problem is the file list itself. The files are ordered by a 4-digit number. So
 
ls -l wind_0*.nc
 
gives me 1000 file names,
 
ls -l wind_[0-2]*.nc
 
results in 3000 file names. All works well with up to 2000 file names. In ferret:
 
yes? let file_list = SPAWN("ls -1 wind_0*nc")
 
yes? list file_list
 
gives the correct list and the aggregation works great as it should.
 
yes? let file_list = SPAWN("ls -1 wind_[0-2]*nc")
 
yes? list file_list
 
results in huge traceback and ferret quits.
 
yes? list file_list
*** Error in `/sw/data/python/miniconda/OS_42.3/2.7.15/bin/python': free(): invalid pointer: 0x00007fcfbc746698 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x740ef)[0x7fcfbc41b0ef]
/lib64/libc.so.6(+0x79646)[0x7fcfbc420646]
/lib64/libc.so.6(+0x7a393)[0x7fcfbc421393]
/sw/viz/ferret/py_ferret_v750_os423_py_2715_netcdf463/lib/python2.7/site-packages/pyferret/libpyferret.so(FerMem_Free+0x1a)[0x7fcfb622ee83]
 
...
 
/sw/data/python/miniconda/OS_42.3/2.7.15/bin/../lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7d14)[0x7fcfbd1b2ea4]
 
... several 100 similar lines from many other libraries in the miniconda based python including a memory map  and finally
 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/sw/viz/ferret/py_ferret_v750_os423_py_2715_netcdf463/lib/python2.7/site-packages/pyferret/__init__.py", line 482, in init
    result = run()
  File "/sw/viz/ferret/py_ferret_v750_os423_py_2715_netcdf463/lib/python2.7/site-packages/pyferret/__init__.py", line 887, in run
    retval = libpyferret._run(str_command)
RuntimeError: etc
 
I have no idea, which limit or ulimit, (but it is not the stacksize) could be violated.  I have no idea how to produce this list in python directly, to check if this a python or ferret issue.
 
in _init_.py it crashs here:
 
        readline.set_completer_delims(FERRET_READLINE_DELIMS)
    # the actual Ferret function call
    retval = libpyferret._run(str_command)
 
Is there eventually a limit in ferret itself?
 
Any hints would be very welcome.
 
Best,
 
Martin
 
 

[Thread Prev][Thread Next][Index]