The pixman library in not part of xrdp. If users see that "xrdp_pixman"
is not found, they might assume it is some xrdp specific library.
The Automake conditional is still XRDP_PIXMAN, but the pkg-config
variables become PIXMAN_LIBS and PIXMAN_CFLAGS.
AM_CPPFLAGS is a documented Automake variable for C preprocessor flags
that should not be overridden when compiling the package.
There is no need to have two additional variables that are ultimately
merged into AM_CPPFLAGS.
Their names are also confusing. EXTDA_DIST is a documented Automake
variable. Everything else that starts with "EXTRA" is not.
libdir is for shared and static libraries. Some xrdp libraries are loaded
dynamically from a non-standard directory, so they are used like modules.
Having separate libdir and moduledir would eventually allow to separate
dynamically loaded modules from shared libraries.
AM_CPPFLAGS is for flags passed to the preprocessor, such as defines and
includes. AM_CFLAGS is for flags affecting the compiler, such as debug
and optimization settings.
INCLUDES is an obsolete name. Users can pass INCLUDES and break
compilation. AM_CPPFLAGS is more explicit that the flags come from
Automake and should not be overridden.
To add flags to the compiler, CFLAGS, CPPFLAGS or LDFLAGS can be used on
the configure command line.
The need to add flags depends on the location of the headers and
libraries of the dependencies, which is orthogonal to the directory where
xrdp will be installed.
The implementation in configure.ac has a stray closing bracket, making
GOT_PREFIX true even if --prefix is not passed.
The implementation is inconsistent - the only affected makefiles are for
xrdp and libxrdp.
Changing rpath manually is wrong in most cases. Libtool should be able to
set rpath correctly on its own.
Using $(prefix)/lib ignores the libdir setting. For many 64-bit systems,
/usr/lib is used for 32-bit libraries. Adding 32-bit libraries to the
rpath slows down 64-bit executables, as the dynamic loader searches for
libraries in a wrong directory.
There is no way to disable GOT_PREFIX if --prefix has to be passed.
Fedora RPM patches configure.ac and needs to rerun autoconf and automake
after that.