[Perldl] A fast and natural interface to R from Perl

Discussion:

Zakariyya Mughal

2014-12-24 02:19:05 UTC

Hi everyone,

I have (finally) uploaded modules for working with the R interpreter
with Perl. The CPAN links are below, but to get a taste of what the API
looks like, check out my blog post <http://enetdown.org/dot-plan/posts/2014/12/24/a_fast_and_natural_interface_to_R_from_Perl/>.

- Statistics::NiceR <http://p3rl.org/Statistics::NiceR>
- Data::Frame <http://p3rl.org/Data::Frame>

I'd love to have feedback on how to improve them.

Regards and happy hacking,
- Zaki Mughal

Chris Marshall

2014-12-24 14:55:46 UTC

Permalink

Very cool! Thanks for expanding the space of perl and PDL computation! In
your work, did you determine anything PDL3 would need to do a better job to
support using R from perl?

--Chris

Post by Zakariyya Mughal
Hi everyone,
I have (finally) uploaded modules for working with the R interpreter
with Perl. The CPAN links are below, but to get a taste of what the API
looks like, check out my blog post <
http://enetdown.org/dot-plan/posts/2014/12/24/a_fast_and_natural_interface_to_R_from_Perl/

- Statistics::NiceR <http://p3rl.org/Statistics::NiceR>
- Data::Frame <http://p3rl.org/Data::Frame>
I'd love to have feedback on how to improve them.
Regards and happy hacking,
- Zaki Mughal
_______________________________________________
Perldl mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl

Zakariyya Mughal

2014-12-24 23:31:17 UTC

Permalink

Post by Chris Marshall
Very cool! Thanks for expanding the space of perl and PDL computation! In
your work, did you determine anything PDL3 would need to do a better job to
support using R from perl?

Sure, there were a couple things that would have been nice to have:

For Data::Frame,

- It's a small thing, but a way to "plug-in" to the stringification for
PDL subclasses would make implementing subclasses easier. Right
now, PDL's `string` method is a bit of a black-box because it
stringifies all the elements at once. Instead, I had to write my own
string1d function [^stringifiable].

- Make a hash-based PDL the default. While using the `initialize` function
combined with `FOREIGNBUILDARGS` is an easy way to get PDL working
with Moo[se], it is extra code [^moo-hash-pdl].

- It might be useful to have annotations of all functions that do not
change the values of elements. I am using that for enum-like data
where I want the levels (the possible values of the enum) to be copied
over to new enum-like PDLs. So I wrap the following methods:

around qw(slice uniq dice) => sub { ... };

but I'm not sure if that covers everything [^around-enum].

My thoughts on this: perhaps the PDL class has too many methods by
default. There should be a way to pare that down using roles, but
deciding what goes in each role does not seem straightforward to me at
this time.

For Statistics::NiceR,

- The way that R stores data is inside a SEXP C structure. You can reach
inside and get at the data by using a macro which points to the memory
address like:

SEXP r_sexp_integer, r_sexp_real;
INTEGER(r_sexp_integer)[ idx ] /* access the int32_t value at idx */

REAL(r_sexp_real)[ idx ] /* access the double value at idx */

Currently, I'm just using memcpy() to get the R data into a PDL. I
haven't used pdl_wrap() on the R data yet, but I plan to soon. But
what I'm wondering is: can I change the way PDL allocates data so that
it will create the R's SEXP C structure in the background — perhaps
limited to a scope? This might be YAGNI, but it might have
implications for things like GPU support. Instead of having to
explicitly create GPU arrays all the time, there should be a way of
indicating that a piece of code will be using a different allocator
than usual.

- Speaking of different allocation types, it might be useful to look at
how other tools extend their built-in types. I'll give some R
examples:

- R's bigmemory <http://cran.r-project.org/web/packages/bigmemory/index.html>,
<http://www.stat.yale.edu/~mjk56/temp/bigmemory-vignette.pdf>,
<http://2013.hpcs.ca/wp-content/uploads/2013/07/HPCS2013-Parallel-Work-with-R.pdf>.

Not only does this support mmap'ed files (like PDL::IO::{FastRaw,FlexRaw}),
but they also have associated packages that have specialised
versions things like linear regression (in biglm) and k-means
clustering (in biganalytics).

- R's GMP <http://cran.r-project.org/web/packages/gmp/index.html>.

It's a wrapper for the GMP library for big integers/rationals, but
it also lets you create matrices of big numbers which can be used
for solving a system of equations (solve.bigz).

[^stringifiable]: Role that lets elements stringify themselves
<https://github.com/zmughal/p5-Data-Frame/blob/master/lib/PDL/Role/Stringifiable.pm>.

[^moo-hash-pdl]: <https://github.com/zmughal/p5-Data-Frame/blob/master/lib/PDL/Factor.pm> has the following code:

use Moo;
extends 'PDL';
around new => sub {
my $orig = shift;
my ($class, @args) = @_;
# snip...
unshift @args, _data => $enum;
my $self = $orig->($class, @args);
# snip...
}

sub FOREIGNBUILDARGS {
my ($self, %args) = @_;
( $args{_data} );
}

sub initialize {
bless { PDL => PDL::null() }, shift;
}

[^around-enum]: <https://github.com/zmughal/p5-Data-Frame/blob/master/lib/PDL/Role/Enumerable.pm#L46>.

Cheers,
- Zaki Mughal

Post by Chris Marshall
--Chris

Chris Marshall

2014-12-29 18:07:56 UTC

Permalink

Thanks for the detailed and referenced response! More below.

--Chris

Post by Zakariyya Mughal

Post by Chris Marshall
Very cool! Thanks for expanding the space of perl and PDL computation!

Post by Chris Marshall
your work, did you determine anything PDL3 would need to do a better job

Post by Chris Marshall
support using R from perl?

For Data::Frame,
- It's a small thing, but a way to "plug-in" to the stringification for
PDL subclasses would make implementing subclasses easier. Right
now, PDL's `string` method is a bit of a black-box because it
stringifies all the elements at once. Instead, I had to write my own
string1d function [^stringifiable].

Good point. This is one of the reasons to start PDL3 from the core out so
that things like stringification can be handled naturally in subclasses.

Post by Zakariyya Mughal
- Make a hash-based PDL the default. While using the `initialize` function
combined with `FOREIGNBUILDARGS` is an easy way to get PDL working
with Moo[se], it is extra code [^moo-hash-pdl].

Yes, and maybe a tweak to the logic for PDL-2.x so that it works seamlessly
with PDL3 stuff.

Post by Zakariyya Mughal
- It might be useful to have annotations of all functions that do not
change the values of elements. I am using that for enum-like data
where I want the levels (the possible values of the enum) to be copied
around qw(slice uniq dice) => sub { ... };
but I'm not sure if that covers everything [^around-enum].

Interesting. It seems like this is meta-info support for PDL specific
operations. I wonder if there are other examples where similar function
meta-data is being used for this type of problem?

Post by Zakariyya Mughal
My thoughts on this: perhaps the PDL class has too many methods by
default. There should be a way to pare that down using roles, but
deciding what goes in each role does not seem straightforward to me at
this time.

The current PDL module is based on the kitchen sink approach for
computation in that all of the standard features are always imported and
made available. There is an existing feature request (or bug) to fix the
import/export handling of PDL so that it matches expected usages for perl
modules. It seems to me that the kitchen sink set of modules is more
appropriate for interactive use rather than programming. We could add
support for that in the PDL shells. On the other hand, with as many
functions as PDL has, it could be a bit of a pain if things are spun out
into enclosing namespaces that are too small.

Post by Zakariyya Mughal
For Statistics::NiceR,
- The way that R stores data is inside a SEXP C structure. You can reach
inside and get at the data by using a macro which points to the memory
SEXP r_sexp_integer, r_sexp_real;
INTEGER(r_sexp_integer)[ idx ] /* access the int32_t value at idx */
REAL(r_sexp_real)[ idx ] /* access the double value at idx */
Currently, I'm just using memcpy() to get the R data into a PDL. I
haven't used pdl_wrap() on the R data yet, but I plan to soon. But
what I'm wondering is: can I change the way PDL allocates data so that
it will create the R's SEXP C structure in the background â perhaps
limited to a scope? This might be YAGNI, but it might have
implications for things like GPU support. Instead of having to
explicitly create GPU arrays all the time, there should be a way of
indicating that a piece of code will be using a different allocator
than usual.

The aspects of PDL3 relevant to this are:
* Improved type support including general, user defined ones
* Make PDL data and computation usable from C or Perl
* PDL-2.x already has some of this in a hand-rolled form

I took a quick look at the R internals and SEXP stuff. It looks a lot like
the R flavor of Perl's SV*

Post by Zakariyya Mughal
- Speaking of different allocation types, it might be useful to look at
how other tools extend their built-in types. I'll give some R
- R's bigmemory <
http://cran.r-project.org/web/packages/bigmemory/index.html>,
<http://www.stat.yale.edu/~mjk56/temp/bigmemory-vignette.pdf>,
<
http://2013.hpcs.ca/wp-content/uploads/2013/07/HPCS2013-Parallel-Work-with-R.pdf

Post by Chris Marshall
.

Not only does this support mmap'ed files (like
PDL::IO::{FastRaw,FlexRaw}),
but they also have associated packages that have specialised
versions things like linear regression (in biglm) and k-means
clustering (in biganalytics).
- R's GMP <http://cran.r-project.org/web/packages/gmp/index.html>.
It's a wrapper for the GMP library for big integers/rationals, but
it also lets you create matrices of big numbers which can be used
for solving a system of equations (solve.bigz).

Thanks for the references. I would like to see PDL's type support improved
and having use cases to exercise against any ideas for the new architecture
are a help.

Post by Zakariyya Mughal
[^stringifiable]: Role that lets elements stringify themselves
<
https://github.com/zmughal/p5-Data-Frame/blob/master/lib/PDL/Role/Stringifiable.pm

Post by Chris Marshall
.

[^moo-hash-pdl]: <
https://github.com/zmughal/p5-Data-Frame/blob/master/lib/PDL/Factor.pm>
use Moo;
extends 'PDL';
around new => sub {
my $orig = shift;
# snip...
# snip...
}
sub FOREIGNBUILDARGS {
( $args{_data} );
}
sub initialize {
bless { PDL => PDL::null() }, shift;
}
[^around-enum]: <
https://github.com/zmughal/p5-Data-Frame/blob/master/lib/PDL/Role/Enumerable.pm#L46

Post by Chris Marshall
.

Cheers,
- Zaki Mughal

Post by Chris Marshall
--Chris

http://enetdown.org/dot-plan/posts/2014/12/24/a_fast_and_natural_interface_to_R_from_Perl/

Post by Chris Marshall

Post by Zakariyya Mughal

Pablo marin-garcia

2014-12-28 00:13:21 UTC

Permalink

Dear Zakariyya,

Thanks for the distribution, I was just looking this week for pdl/perl
distributions to work with dataframes I use both in R and Perl. I was
looking for something in pdl or Perl for doing dataframe subselections and
calculations. I can use pdl for numbers (in the matrix part of dataframes)
and for strings that are all of the same size with PDL::Char but was
complicated to subselect by alphanumeric factors without prior
'numerification' .

I hope your modules would be a good option for this functionality but
unfortunately I was unable to install ithem at the first go.

Here I report my installation errors just in case are of any help

Installation error report:
------------------------------

Note1: I am reporting this here just in case if of any help to you.

Note2: R does not come from a package but instead is compiled by myself.

There is a recurrent error in each of the module test but test is reported
OK ??!!

unable to load shared object
'/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so':
libRlapack.so: cannot open shared object file: No such file or
directory
Package âstatsâ in options("defaultPackages") was not found
t/conversion.t .............. ok

But stats.so exists:

$ locate stats.so
/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so
--------
# intallation command
$ cpanm Statistics::NiceR

Building and testing Statistics-NiceR-0.03 ... *** stack smashing detected
***: /home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl terminated

======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x2b222e805807]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x0)[0x2b222e8057d0]
/soft_bio/R/R-3.0.2/lib/libR.so(+0xef05e)[0x2b22306e805e]
[0x40f3648]
======= Memory map: ========
00400000-00555000 r-xp 00000000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
00755000-00756000 r--p 00155000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
00756000-0075b000 rw-p 00156000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
0075b000-0075c000 rw-p 00000000 00:00 0
01d17000-04c45000 rw-p 00000000 00:00 0
[heap]
2b222dd9c000-2b222ddbe000 r-xp 00000000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222ddbe000-2b222ddc0000 rw-p 00000000 00:00 0
2b222dfbe000-2b222dfbf000 r--p 00022000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222dfbf000-2b222dfc1000 rw-p 00023000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222dfc1000-2b222dfc3000 r-xp 00000000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222dfc3000-2b222e1c3000 ---p 00002000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c3000-2b222e1c4000 r--p 00002000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c4000-2b222e1c5000 rw-p 00003000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c5000-2b222e2c0000 r-xp 00000000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e2c0000-2b222e4bf000 ---p 000fb000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4bf000-2b222e4c0000 r--p 000fa000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4c0000-2b222e4c1000 rw-p 000fb000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4c1000-2b222e4c2000 rw-p 00000000 00:00 0
2b222e4c2000-2b222e4cb000 r-xp 00000000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e4cb000-2b222e6cb000 ---p 00009000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cb000-2b222e6cc000 r--p 00009000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cc000-2b222e6cd000 rw-p 0000a000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cd000-2b222e6fb000 rw-p 00000000 00:00 0
2b222e6fb000-2b222e8b0000 r-xp 00000000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222e8b0000-2b222eaaf000 ---p 001b5000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eaaf000-2b222eab3000 r--p 001b4000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eab3000-2b222eab5000 rw-p 001b8000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eab5000-2b222eabc000 rw-p 00000000 00:00 0
2b222eabc000-2b222ed85000 r--p 00000000 fc:00 1311449
/usr/lib/locale/locale-archive
2b222ed85000-2b222ed90000 r-xp 00000000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ed90000-2b222ef8f000 ---p 0000b000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef8f000-2b222ef90000 r--p 0000a000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef90000-2b222ef91000 rw-p 0000b000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef91000-2b222ef92000 r--p 00000000 fc:00 267913
/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo
2b222ef92000-2b222ef97000 r-xp 00000000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222ef97000-2b222f196000 ---p 00005000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f196000-2b222f197000 r--p 00004000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f197000-2b222f198000 rw-p 00005000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f198000-2b222f19f000 r-xp 00000000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f19f000-2b222f39e000 ---p 00007000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f39e000-2b222f39f000 r--p 00006000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f39f000-2b222f3a0000 rw-p 00007000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f3a0000-2b222f3b8000 r-xp 00000000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f3b8000-2b222f5b7000 ---p 00018000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b7000-2b222f5b8000 r--p 00017000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b8000-2b222f5b9000 rw-p 00018000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b9000-2b222f5bd000 rw-p 00000000 00:00 0
2b222f5bd000-2b222f5c1000 r-xp 00000000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
2b222f5c1000-2b222f7c0000 ---p 00004000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
2b222f7c0000-2b222f7c1000 r--p 00003000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so

== The output of cpanm build.log

[...]

Building and testing Statistics-NiceR-0.03

Error in dyn.load(file, DLLpath = DLLpath, ...) :

unable to load shared object
'/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so':

libRlapack.so: cannot open shared object file: No such file or directory

During startup - Warning message:

package âstatsâ in options("defaultPackages") was not found

Manifying 13 pod documents

Error in dyn.load(file, DLLpath = DLLpath, ...) :

unable to load shared object
'/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so':

libRlapack.so: cannot open shared object file: No such file or directory

During startup - Warning message:

package âstatsâ in options("defaultPackages") was not found

PERL_DL_NONLAZY=1 "/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl"
"-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef
*Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t

Error in dyn.load(file, DLLpath = DLLpath, ...) :

unable to load shared object
'/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so':

libRlapack.so: cannot open shared object file: No such file or directory

During startup - Warning message:

package âstatsâ in options("defaultPackages") was not found

t/conversion.t .............. ok

Error in dyn.load(file, DLLpath = DLLpath, ...) :

unable to load shared object
'/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so':

libRlapack.so: cannot open shared object file: No such file or directory

During startup - Warning message:

package âstatsâ in options("defaultPackages") was not found

t/equality.t ................ ok

[...]

Error in dyn.load(file, DLLpath = DLLpath, ...) :

unable to load shared object
'/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so':

libRlapack.so: cannot open shared object file: No such file or directory

During startup - Warning message:

package âstatsâ in options("defaultPackages") was not found

Error: could not find function "DESTROY"

In addition: Warning message:

In mean.default(X[[5L]], ...) :

argument is not numeric or logical: returning NA

t/lapply.t .................. ok

Error in dyn.load(file, DLLpath = DLLpath, ...) :

unable to load shared object
'/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so':

libRlapack.so: cannot open shared object file: No such file or directory

During startup - Warning message:

package âstatsâ in options("defaultPackages") was not found

Error: negative length vectors are not allowed

*** caught segfault ***

address (nil), cause 'memory not mapped'

Possible actions:

1: abort (with core dump, if enabled)

2: normal R exit

3: exit R without saving workspace

4: exit R saving workspace

make: *** [test_dynamic] Interrupt

make: *** [test_dynamic] Interrupt
=================================

Post by Zakariyya Mughal
Hi everyone,
I have uploaded modules for working with the R interpreter from Perl.
The CPAN links are below, but to get a taste of what the API
looks like, check out my blog post <
http://enetdown.org/dot-plan/posts/2014/12/24/a_fast_and_natural_interface_to_R_from_Perl/

- Statistics::NiceR <http://p3rl.org/Statistics::NiceR>
- Data::Frame <http://p3rl.org/Data::Frame>
I'd love to have feedback on how to improve them.
Regards and happy hacking,
- Zaki Mughal
_______________________________________________
Bioperl-l mailing list
http://mailman.open-bio.org/mailman/listinfo/bioperl-l

Zakariyya Mughal

2015-02-07 06:09:40 UTC

Permalink

Post by Pablo marin-garcia
Dear Zakariyya,
Thanks for the distribution, I was just looking this week for pdl/perl
distributions to work with dataframes I use both in R and Perl. I was
looking for something in pdl or Perl for doing dataframe subselections and
calculations. I can use pdl for numbers (in the matrix part of dataframes)
and for strings that are all of the same size with PDL::Char but was
complicated to subselect by alphanumeric factors without prior
'numerification' .
I hope your modules would be a good option for this functionality but
unfortunately I was unable to install ithem at the first go.

I haven't forgot about this bug! I am still looking into debugging it,
but from what I have seen so far, this stack smashing error only happens
while running under the test harness. I don't know why, but at least
that is the first step towards reproducing it.

I'm not sure about the issue with the shared object. I think that is
from how R gets loaded using R's RHOME, but I don't know how to
reproduce it. What do

R RHOME

R CMD config --cppflags

R CMD config --ldflags

ldd `R RHOME`/bin/exec/R

give as output?

I suspect that the warnings can be fixed by making sure that the library
is loaded ahead of time:

export LD_PRELOAD='/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so'
cpanm --verbose Statistics::NiceR

but that's not optimal.

The library should be fine to use otherwise. Please let me know if this
helps.

Regards,
- Zaki Mughal

Post by Pablo marin-garcia
Here I report my installation errors just in case are of any help
------------------------------
Note1: I am reporting this here just in case if of any help to you.
Note2: R does not come from a package but instead is compiled by myself.
There is a recurrent error in each of the module test but test is reported
OK ??!!
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
Package ‘stats’ in options("defaultPackages") was not found
t/conversion.t .............. ok
$ locate stats.so
/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so
--------
# intallation command
$ cpanm Statistics::NiceR
Building and testing Statistics-NiceR-0.03 ... *** stack smashing detected
***: /home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x2b222e805807]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x0)[0x2b222e8057d0]
/soft_bio/R/R-3.0.2/lib/libR.so(+0xef05e)[0x2b22306e805e]
[0x40f3648]
======= Memory map: ========
00400000-00555000 r-xp 00000000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
00755000-00756000 r--p 00155000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
00756000-0075b000 rw-p 00156000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
0075b000-0075c000 rw-p 00000000 00:00 0
01d17000-04c45000 rw-p 00000000 00:00 0
[heap]
2b222dd9c000-2b222ddbe000 r-xp 00000000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222ddbe000-2b222ddc0000 rw-p 00000000 00:00 0
2b222dfbe000-2b222dfbf000 r--p 00022000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222dfbf000-2b222dfc1000 rw-p 00023000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222dfc1000-2b222dfc3000 r-xp 00000000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222dfc3000-2b222e1c3000 ---p 00002000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c3000-2b222e1c4000 r--p 00002000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c4000-2b222e1c5000 rw-p 00003000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c5000-2b222e2c0000 r-xp 00000000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e2c0000-2b222e4bf000 ---p 000fb000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4bf000-2b222e4c0000 r--p 000fa000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4c0000-2b222e4c1000 rw-p 000fb000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4c1000-2b222e4c2000 rw-p 00000000 00:00 0
2b222e4c2000-2b222e4cb000 r-xp 00000000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e4cb000-2b222e6cb000 ---p 00009000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cb000-2b222e6cc000 r--p 00009000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cc000-2b222e6cd000 rw-p 0000a000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cd000-2b222e6fb000 rw-p 00000000 00:00 0
2b222e6fb000-2b222e8b0000 r-xp 00000000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222e8b0000-2b222eaaf000 ---p 001b5000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eaaf000-2b222eab3000 r--p 001b4000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eab3000-2b222eab5000 rw-p 001b8000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eab5000-2b222eabc000 rw-p 00000000 00:00 0
2b222eabc000-2b222ed85000 r--p 00000000 fc:00 1311449
/usr/lib/locale/locale-archive
2b222ed85000-2b222ed90000 r-xp 00000000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ed90000-2b222ef8f000 ---p 0000b000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef8f000-2b222ef90000 r--p 0000a000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef90000-2b222ef91000 rw-p 0000b000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef91000-2b222ef92000 r--p 00000000 fc:00 267913
/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo
2b222ef92000-2b222ef97000 r-xp 00000000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222ef97000-2b222f196000 ---p 00005000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f196000-2b222f197000 r--p 00004000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f197000-2b222f198000 rw-p 00005000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f198000-2b222f19f000 r-xp 00000000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f19f000-2b222f39e000 ---p 00007000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f39e000-2b222f39f000 r--p 00006000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f39f000-2b222f3a0000 rw-p 00007000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f3a0000-2b222f3b8000 r-xp 00000000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f3b8000-2b222f5b7000 ---p 00018000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b7000-2b222f5b8000 r--p 00017000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b8000-2b222f5b9000 rw-p 00018000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b9000-2b222f5bd000 rw-p 00000000 00:00 0
2b222f5bd000-2b222f5c1000 r-xp 00000000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
2b222f5c1000-2b222f7c0000 ---p 00004000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
2b222f7c0000-2b222f7c1000 r--p 00003000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
== The output of cpanm build.log
[...]
Building and testing Statistics-NiceR-0.03
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
Manifying 13 pod documents
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
PERL_DL_NONLAZY=1 "/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl"
"-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef
*Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
t/conversion.t .............. ok
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
t/equality.t ................ ok
[...]
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
Error: could not find function "DESTROY"
argument is not numeric or logical: returning NA
t/lapply.t .................. ok
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
Error: negative length vectors are not allowed
*** caught segfault ***
address (nil), cause 'memory not mapped'
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
make: *** [test_dynamic] Interrupt
make: *** [test_dynamic] Interrupt
=================================

- Statistics::NiceR <http://p3rl.org/Statistics::NiceR>
- Data::Frame <http://p3rl.org/Data::Frame>
I'd love to have feedback on how to improve them.
Regards and happy hacking,
- Zaki Mughal
_______________________________________________
Bioperl-l mailing list
http://mailman.open-bio.org/mailman/listinfo/bioperl-l

_______________________________________________
Perldl mailing list
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl

Zakariyya Mughal

2014-12-28 05:24:30 UTC

Permalink

Post by Pablo marin-garcia
Dear Zakariyya,
Thanks for the distribution, I was just looking this week for pdl/perl
distributions to work with dataframes I use both in R and perl. I was
looking for something in pdl or perl for doing dataframe subselections and
calculations. I can use pdl for numbers (in the matrix part of dataframes)
or for strings all of the same size with PDL::Char but was complicated to
subselect by alphanumeric factors without prior 'numerification' .
I hope your modules would be a good option for this functionality but
unfortunately I was unable to install ithem at the first go.

Sorry for the late reply. I was out of town for a few days. Apologies
for the crossposting.

Hmm, the issue with the "missing" stats package could stem from multiple
build problems. I see that some say it could have to do with having
multiple architectures or multiple versions of shared libraries, but
I'll need to try to reproduce it. I'm going to try to build my own R in
a VM and see if it can be fixed. I've been working on making the build
more robust: so far I have made Statistics::NiceR build on Windows under
Strawberry Perl with the default R + Rtools installers. Hopefully what I
learned from that can help me figure this out.

You can get data frames and factors using just the code in Data::Frame,
but I'm still working on the documentation and the API. I'm trying to
get the best ideas out of different tools and providing comfortable
interfaces. Let me know how you would like to use the data structures
and what functions are missing and I'll implement them (e.g., I just
wrote some head()/tail() functions for data frames that'll I'll release
tonight.).

Cheers,
- Zaki Mughal

Post by Pablo marin-garcia
------------------------------
Note1: I am reporting this here just in case if of any help to you.
Note2: R does not come from a package but instead is compiled by myself.
There is a recurrent error in each of the module test but test is reported
OK ??!!
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or
directory
Package ‘stats’ in options("defaultPackages") was not found
t/conversion.t .............. ok
$ locate stats.so
/storage/ethernus_miscellanea/scratch_local/ugdg_software/R/R-3.0.2/library/stats/libs/stats.so
--------
# intallation command
$ cpanm Statistics::NiceR
Building and testing Statistics-NiceR-0.03 ... *** stack smashing detected
***: /home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x2b222e805807]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x0)[0x2b222e8057d0]
/soft_bio/R/R-3.0.2/lib/libR.so(+0xef05e)[0x2b22306e805e]
[0x40f3648]
======= Memory map: ========
00400000-00555000 r-xp 00000000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
00755000-00756000 r--p 00155000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
00756000-0075b000 rw-p 00156000 fc:00 793010
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl
0075b000-0075c000 rw-p 00000000 00:00 0
01d17000-04c45000 rw-p 00000000 00:00 0
[heap]
2b222dd9c000-2b222ddbe000 r-xp 00000000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222ddbe000-2b222ddc0000 rw-p 00000000 00:00 0
2b222dfbe000-2b222dfbf000 r--p 00022000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222dfbf000-2b222dfc1000 rw-p 00023000 fc:00 1186920
/lib/x86_64-linux-gnu/ld-2.15.so
2b222dfc1000-2b222dfc3000 r-xp 00000000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222dfc3000-2b222e1c3000 ---p 00002000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c3000-2b222e1c4000 r--p 00002000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c4000-2b222e1c5000 rw-p 00003000 fc:00 1186905
/lib/x86_64-linux-gnu/libdl-2.15.so
2b222e1c5000-2b222e2c0000 r-xp 00000000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e2c0000-2b222e4bf000 ---p 000fb000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4bf000-2b222e4c0000 r--p 000fa000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4c0000-2b222e4c1000 rw-p 000fb000 fc:00 1186914
/lib/x86_64-linux-gnu/libm-2.15.so
2b222e4c1000-2b222e4c2000 rw-p 00000000 00:00 0
2b222e4c2000-2b222e4cb000 r-xp 00000000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e4cb000-2b222e6cb000 ---p 00009000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cb000-2b222e6cc000 r--p 00009000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cc000-2b222e6cd000 rw-p 0000a000 fc:00 1186915
/lib/x86_64-linux-gnu/libcrypt-2.15.so
2b222e6cd000-2b222e6fb000 rw-p 00000000 00:00 0
2b222e6fb000-2b222e8b0000 r-xp 00000000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222e8b0000-2b222eaaf000 ---p 001b5000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eaaf000-2b222eab3000 r--p 001b4000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eab3000-2b222eab5000 rw-p 001b8000 fc:00 1186906
/lib/x86_64-linux-gnu/libc-2.15.so
2b222eab5000-2b222eabc000 rw-p 00000000 00:00 0
2b222eabc000-2b222ed85000 r--p 00000000 fc:00 1311449
/usr/lib/locale/locale-archive
2b222ed85000-2b222ed90000 r-xp 00000000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ed90000-2b222ef8f000 ---p 0000b000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef8f000-2b222ef90000 r--p 0000a000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef90000-2b222ef91000 rw-p 0000b000 fc:00 822259
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/site_perl/5.16.0/x86_64-linux/auto/List/Util/Util.so
2b222ef91000-2b222ef92000 r--p 00000000 fc:00 267913
/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo
2b222ef92000-2b222ef97000 r-xp 00000000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222ef97000-2b222f196000 ---p 00005000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f196000-2b222f197000 r--p 00004000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f197000-2b222f198000 rw-p 00005000 fc:00 1183229
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/Time/HiRes/HiRes.so
2b222f198000-2b222f19f000 r-xp 00000000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f19f000-2b222f39e000 ---p 00007000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f39e000-2b222f39f000 r--p 00006000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f39f000-2b222f3a0000 rw-p 00007000 fc:00 1186910
/lib/x86_64-linux-gnu/librt-2.15.so
2b222f3a0000-2b222f3b8000 r-xp 00000000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f3b8000-2b222f5b7000 ---p 00018000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b7000-2b222f5b8000 r--p 00017000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b8000-2b222f5b9000 rw-p 00018000 fc:00 1186908
/lib/x86_64-linux-gnu/libpthread-2.15.so
2b222f5b9000-2b222f5bd000 rw-p 00000000 00:00 0
2b222f5bd000-2b222f5c1000 r-xp 00000000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
2b222f5c1000-2b222f7c0000 ---p 00004000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
2b222f7c0000-2b222f7c1000 r--p 00003000 fc:00 1183155
/home/pmg/perl5/perlbrew/perls/perl-5.16.0/lib/5.16.0/x86_64-linux/auto/mro/mro.so
== The output of cpanm build.log
[...]
Building and testing Statistics-NiceR-0.03
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
Manifying 13 pod documents
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
PERL_DL_NONLAZY=1 "/home/pmg/perl5/perlbrew/perls/perl-5.16.0/bin/perl"
"-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef
*Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
t/conversion.t .............. ok
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
t/equality.t ................ ok
[...]
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
Error: could not find function "DESTROY"
argument is not numeric or logical: returning NA
t/lapply.t .................. ok
unable to load shared object
libRlapack.so: cannot open shared object file: No such file or directory
package ‘stats’ in options("defaultPackages") was not found
Error: negative length vectors are not allowed
*** caught segfault ***
address (nil), cause 'memory not mapped'
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
make: *** [test_dynamic] Interrupt
make: *** [test_dynamic] Interrupt

- Statistics::NiceR <http://p3rl.org/Statistics::NiceR>
- Data::Frame <http://p3rl.org/Data::Frame>
I'd love to have feedback on how to improve them.
Regards and happy hacking,
- Zaki Mughal
_______________________________________________
Bioperl-l mailing list
http://mailman.open-bio.org/mailman/listinfo/bioperl-l

Continue reading on narkive:

Search results for '[Perldl] A fast and natural interface to R from Perl' (Questions and Answers)

replies

whats the easyest programming laguage?

started 2009-10-19 20:24:07 UTC

programming & design

102

replies

What is nanotechnology?

started 2006-10-06 15:15:50 UTC

engineering