Multiple Flat Networks on a single physical network

Problem description
===================

Many DC will provide internet access from different providers (or different
subnets) on the same wire, without any kind of tunneling or vlan tagging.
If the deployer wants to allow public access to it's VMs he/she could use
provider network.

Today is not possible to setup two or more network over the same
physical link using flat provider.

If multiple flat networks are allow on the same physical network, deployer
can easily provide internet connectivity to it's customers, using one or more
upstream networks.

Proposed change
===============

Allow to create more than one flat network over a single physical network.
We will allow to override the default of one flat network per physical if a
setting is configured.
The default value for the config setting will be False, that's it keep the
current behavior.

Alternatives
------------
Keep doing the mess we are doing now.

example:
Put the nic on a bridge with two veth and create a second bridge
with each veth and configuring that second bridge as physical interface.

asciiflow::

                                +----------------------------+
                                | |
                                +--------+ |
                             +--> ethA_1 | |
  +------------------------+ | +--------+ br-phy1 |
  | br-proxy +--------+ | | |
  | |ethA_0 +-+ | |
  +------------+ +--------+ +----------------------------+
  | NIC | |
  | | +--------+ +----------------------------+
  +------------+ |ethB_0 +-+ | |
  | +--------+ | | |
  +------------------------+ | +--------+ |
                             +--> ethB_1 | br-phy2 |
                                +--------+ |
                                | |
                                +----------------------------+

Data model impact
-----------------
This change won't impact data model.

REST API impact
---------------
No REST API impact.

Security impact
---------------
It could be say that allowing different networks on the same physical link
(without any encapsulation) is dangerous, as it does not provide any kind
of isolation. Anyway it is an standard industry practice.

Neutron security group avoids sniffing and arp poisoning. So VMs won't
be able to 'steal' others ip or sniff its traffic.

Notifications impact
--------------------
No impact.

Other end user impact
---------------------
No impact to the end user.

Performance Impact
------------------
Code will have to pass more if statments any time a network is created.

Other deployer impact
---------------------
No deployer impact, those who want to use the new feature will set option as
True.

Those who don't want the feature or are upgrading from previous release (or
doing CD) won't have to change their configs as the default is to have the
feature disabled.

Developer impact
----------------
No developer impact.

Implementation
==============

Assignee(s)
-----------
gustavo panizzo <gfa>

I'm looking for guidance from somebody experienced on neutron internals.

Work Items
----------
Single work item.

Dependencies
============

No dependencies.

Testing
=======

A tempest test to test both True and False on new config variable will be
provided.

Documentation Impact
====================

If current documentation makes visible the imposibilty of creating more than
one flat network on a single physical network it will be updated.

References
==========

None.