Percona Server 5.7.11-4 misses the innodb_numa_interleave server variable

It doesn't seem to be a build issue since the build slaves have build dependencies and the cmake output looks like:
-- Looking for include file numaif.h
-- Looking for include file numaif.h - found
-- Performing Test HAVE_LIBNUMA
-- Performing Test HAVE_LIBNUMA - Success
I have checked all the builds for 5.7.11-4 release and didn't find different output than this.

The cmake cache also has:
root@vps-wheezy-x64-02# grep NUMA CMakeCache.txt
//Explicitly set NUMA memory allocation policy
WITH_NUMA:BOOL=ON
//Test HAVE_LIBNUMA
HAVE_LIBNUMA:INTERNAL=1
HAVE_NUMAIF_H:INTERNAL=1

The issue doesn't seem to be related to any specific linux distribution as I checked debian jessie and centos 6 and it's the same.

The regressing upstream commit https://github.com/mysql/mysql-server/commit/5d467636d7a86a40ac5d7edef29144ac4067e48f replaced #ifdef HAVE_LIBNUMA checks with #if defined(HAVE_LIBNUMA) && defined(WITH_NUMA) ones.

Both symbols are defined by CMake, but WITH_NUMA is not passed to C++ preprocessor. It seems that config.h.cmake is missing

#cmakedefine WITH_NUMA 1

Tomislav, can you check whether that fixes it?

Laurynas,
I get an error with that so it looks broken in other place also:
/home/vagrant/build-flash/ps/percona-server/storage/innobase/buf/buf0buf.cc: In constructor ‘set_numa_interleave_t::set_numa_interleave_t()’:
/home/vagrant/build-flash/ps/percona-server/storage/innobase/buf/buf0buf.cc:88:7: error: ‘srv_numa_interleave’ was not declared in this scope
   if (srv_numa_interleave) {
       ^
/home/vagrant/build-flash/ps/percona-server/storage/innobase/buf/buf0buf.cc: In destructor ‘set_numa_interleave_t::~set_numa_interleave_t()’:
/home/vagrant/build-flash/ps/percona-server/storage/innobase/buf/buf0buf.cc:105:7: error: ‘srv_numa_interleave’ was not declared in this scope
   if (srv_numa_interleave) {
       ^
make[2]: *** [extra/CMakeFiles/innochecksum.dir/__/storage/innobase/buf/buf0buf.cc.o] Error 1

It looks like the upstream commit that introduced the regression was not built even once successfully.

What happens if you replace
#if defined(HAVE_LIBNUMA) && defined(WITH_NUMA)
at buf0buf.cc:80 with
#if defined(HAVE_LIBNUMA) && defined(WITH_NUMA) && !defined(UNIV_INNOCHECKSUM)

?

Then it seems to work - it compiles and I can see the variable.
DEFAULT is off:
mysql> show VARIABLES like '%numa%';
+------------------------+---------+
| Variable_name | Value |
|------------------------+---------|
| innodb_numa_interleave | OFF |
+------------------------+---------+

When set in my.cnf:
mysql> show VARIABLES like '%numa%';
+------------------------+---------+
| Variable_name | Value |
|------------------------+---------|
| innodb_numa_interleave | ON |
+------------------------+---------+

The 5.6 seems unaffected as the commit that introduced it is not applied to 5.6 branch.
Here's how it looks:
DEFAULT
mysql> show global variables like '%numa%';
+------------------------+---------+
| Variable_name | Value |
|------------------------+---------|
| innodb_numa_interleave | OFF |
+------------------------+---------+

CHANGED:
mysql> show variables like '%numa%';
+------------------------+---------+
| Variable_name | Value |
|------------------------+---------|
| innodb_numa_interleave | ON |
+------------------------+---------+

VERSION INFO:
mysql> show variables like '%version%';
+-------------------------+------------------------------------------------------+
| Variable_name | Value |
|-------------------------+------------------------------------------------------|
| innodb_version | 5.6.29-rel76.2 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 5.6.29-76.2 |
| version_comment | Percona Server (GPL), Release 76.2, Revision ddf26fe |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
+-------------------------+------------------------------------------------------+

Although there is this bug in upstream for 5.6: http://bugs.mysql.com/bug.php?id=78953
But maybe it's related to their infrastructure or something.

https://github.com/percona/percona-server/pull/483

I am also experiencing this on Percona 5.6.29-76.2 under Red Hat Enterprise Linux Server release 5.8 (Tikanga):

mysql> show variables like '%numa%';
Empty set (0.00 sec)

mysql> show variables like '%version%';
+--------------------------+------------------------------------------------------+
| Variable_name | Value |
+--------------------------+------------------------------------------------------+
| audit_offsets_by_version | ON |
| innodb_version | 5.6.29-76.2 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 5.6.29-76.2-log |
| version_comment | Percona Server (GPL), Release 76.2, Revision ddf26fe |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
+--------------------------+------------------------------------------------------+
8 rows in set (0.00 sec)

Any chance to get it fixed in 5.6 branch?

Tomislav's output in comment #6 shows exact same version unaffected by this bug. Which binaries are you using?

I'm using the package downloaded from Percona site:

$ rpm -qa | grep Percona
Percona-Server-server-56-5.6.29-rel76.2.el5.x86_64

I've now tried and the variable is not visible on 5.6.30-76.3 but only on centos 5. My previous test was on centos 6. There must be something specific with centos 5 distro or our build nodes for centos 5 don't have everything needed.
I see in my vm there is numactl package installed and I can remove it and it doesn't see the connection with Percona Server (where on centos 6 it does and I cannot remove numactl and leave PS on the system).

CENTOS 5:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.30-76.3 Percona Server (GPL), Release 76.3, Revision 3850db5

Copyright (c) 2009-2016 Percona LLC and/or its affiliates
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show variables like '%numa%';
Empty set (0.01 sec)

CENTOS 6:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.30-76.3 Percona Server (GPL), Release 76.3, Revision 3850db5

Copyright (c) 2009-2016 Percona LLC and/or its affiliates
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show variables like '%numa%';
+------------------------+-------+
| Variable_name | Value |
+------------------------+-------+
| innodb_numa_interleave | OFF |
+------------------------+-------+
1 row in set (0.02 sec)

From build nodes:
centos 5:
00:04:32.006 -- Looking for include file numaif.h
00:04:32.290 -- Looking for include file numaif.h - found
00:04:32.309 -- Performing Test HAVE_LIBNUMA
00:04:32.984 -- Performing Test HAVE_LIBNUMA - Failed

centos 6:
00:01:10.492 -- Looking for include file numaif.h
00:01:10.620 -- Looking for include file numaif.h - found
00:01:10.620 -- Performing Test HAVE_LIBNUMA
00:01:10.777 -- Performing Test HAVE_LIBNUMA - Success

I think you might be missing the following on the Centos 5 build node:

yum install numactl-devel

Would you mind giving it a try?

The numactl-devel package is installed there:
[tomislav.plavcic@vps-centos5-x64-04 ~]$ sudo yum list installed|grep "numa"
numactl.i386 0.9.8-12.el5_6 installed
numactl.x86_64 0.9.8-12.el5_6 installed
numactl-devel.i386 0.9.8-12.el5_6 installed
numactl-devel.x86_64 0.9.8-12.el5_6 installed

That package provides the numaif.h so it would fail on including it and it doesn't but cmake test fails - so either the version of the file is too old or there's some other issue why HAVE_LIBNUMA test fails.
Anyway needs more investigation.

is it possible the Centos 5 build node doesn't contain the fix to config.h.cmake and buf0buf.cc from https://github.com/percona/percona-server/pull/483/files?

  config.h.cmake
 @@ -435,6 +435,7 @@
  #cmakedefine DEFAULT_SECURE_FILE_PRIV_DIR @DEFAULT_SECURE_FILE_PRIV_DIR@
  #cmakedefine DEFAULT_SECURE_FILE_PRIV_EMBEDDED_DIR @DEFAULT_SECURE_FILE_PRIV_EMBEDDED_DIR@
  #cmakedefine HAVE_LIBNUMA 1
 +#cmakedefine WITH_NUMA 1

  /* For default value of --early_plugin_load */
  #cmakedefine DEFAULT_EARLY_PLUGIN_LOAD @DEFAULT_EARLY_PLUGIN_LOAD@

 storage/innobase/buf/buf0buf.cc
 @@ -77,7 +77,7 @@ Created 11/5/1995 Heikki Tuuri
  #include <map>
  #include <sstream>

 -#if defined(HAVE_LIBNUMA) && defined(WITH_NUMA)
 +#if defined(HAVE_LIBNUMA) && defined(WITH_NUMA) && !defined(UNIV_INNOCHECKSUM)
  #include <numa.h>
  #include <numaif.h>

Hardly, because all the nodes and distros build from the same source - so if that was the case then it wouldn't work on any, my current best guess is that the version of the library is too old on centos 5 but need to take some time to check why the test fails in the first place.

Thank you Tomislav. I tried to build on Centos5 and here is what I got on CMakeFiles/CMakeError.log:

Run Build Command:/usr/bin/gmake "cmTryCompileExec/fast"
/usr/bin/gmake -f CMakeFiles/cmTryCompileExec.dir/build.make CMakeFiles/cmTryCompileExec.dir/build
gmake[1]: Entering directory `/vagrant/percona-server/CMakeFiles/CMakeTmp'
/usr/bin/cmake -E cmake_progress_report /vagrant/percona-server/CMakeFiles/CMakeTmp/CMakeFiles 1
Building C object CMakeFiles/cmTryCompileExec.dir/src.c.o
/usr/bin/gcc -D_LARGEFILE_SOURCE -D_LARGE_FILES -D_FILE_OFFSET_BITS=64 -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -fPIC -DHAVE_LIBNUMA -o CMakeFiles/cmTryCompileExec.dir/src.c.o -c /vagrant/percona-server/CMakeFiles/CMakeTmp/src.c
/vagrant/percona-server/CMakeFiles/CMakeTmp/src.c: In function ?~@~Xmain?~@~Y:
/vagrant/percona-server/CMakeFiles/CMakeTmp/src.c:6: error: ?~@~Xnuma_all_nodes_ptr?~@~Y undeclared (first use in this function)
/vagrant/percona-server/CMakeFiles/CMakeTmp/src.c:6: error: (Each undeclared identifier is reported only once
/vagrant/percona-server/CMakeFiles/CMakeTmp/src.c:6: error: for each function it appears in.)
gmake[1]: *** [CMakeFiles/cmTryCompileExec.dir/src.c.o] Error 1
gmake[1]: Leaving directory `/vagrant/percona-server/CMakeFiles/CMakeTmp'
gmake: *** [cmTryCompileExec/fast] Error 2

Source file was:

    #include <numa.h>
    #include <numaif.h>
    int main()
    {
       struct bitmask *all_nodes= numa_all_nodes_ptr;
       set_mempolicy(MPOL_DEFAULT, 0, 0);
       return all_nodes != NULL;
    }

So digging a bit more,
On Centos 6
package numactl-devel-2.0.9-2.el6.x86_64
/usr/include/numa.h contains:

/* Set with all nodes from which the calling process may allocate memory.
   Only valid after numa_available. */
extern struct bitmask *numa_all_nodes_ptr;

On Centos 5
package numactl-devel-0.9.8-12.el5_6
/usr/include/numa.h doesn't have any mention of numa_all_nodes_ptr, instead there is:

/* Set with all nodes. Only valid after numa_available. */
extern const nodemask_t numa_all_nodes;

so the cmake check is not passing on Centos5.

Any chance of changing or removing the cmake check?

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-974