Re: guidance on avoiding too many people from one company/group

[Donald Eastlake 3rd <dee3@torque.pothole.com>]

On Sun, 10 Nov 2002, Dave Crocker wrote:

> Date: Sun, 10 Nov 2002 09:28:54 -0800
> From: Dave Crocker <dhc@dcrocker.net>
> To: avri doria <avri@sm.luth.se>
> Cc: ietf-nomcom <ietf-nomcom@lists.elistx.com>
> Subject: Re: guidance on avoiding too many people from one company/group
> 
> Avri,
> 
> Nice summary.
> 
> Nomcom differs from working group processes in two ways: its deliberations
> are closed and the damage likely from reversing the results of those
> deliberations is very high. This prompts us to look for ways to make
> reversal very, very unlikely. Hence, the nomcom processes need to have more
> "surface" objectivity than the usual, open, lengthy incremental IETF
> processes.

I think the main problem is the appearance of fairness. Perfect fairness 
in not actually necessary in any particular year, because random 
selection of the nomcom will tend to smooth things out over time. But 
too great a deviation from the appearance of fairness in one year is 
unacceptable.

The nomcom has an exact, enumerated, publicly known membership. Because
of this and because of the effects on its decisions, it is also much
more likely that its membership will be scrutinized both within and
without the IETF.

> We rely on the diversity achieve this. That is, we rely on random sampling
> to produce a group from sufficiently diverse backgrounds, affiliations,
> locations, etc. This will ensure a sufficiently diverse range of agendas and
> perspectives. (We are not simply worried about conscious and manipulative
> agendas. We are also worried that the group be able to consider things
> broadly. Commonality of a characteristic -- like company affiliation --
> makes it more likely that people will see and analyze things in a similar
> fashion. For a nomcom, this would be problematic.

You are waving your hands vaguely when that isn't necessary. The IETF
is, and is intended to be, a technocratic orgnaization. The problem is
that organizations, even non-profit ones, are sometimes biased
technically. So if persons beholden to one organization get more than
two votes on the nomcom, it looks like, and might actually be, that the
nomcom would be biased towards candidates who would be biased towards
that organization or to technology favored by that organization. This is
the only problem I see, not some vague necessity to be abstractly
diverse for all dimensions (an obvious impossibility).

> Unfortunately the selection process must rely on a rather small
> "population", namely folks who self-select to offer to participate. (The
> pool of candidate nomcom participants is inherently biased, because it is
> self-selected and because even as many as 100 volunteers makes for
> statistically problematic pool.) So it is not suprising that -- as we have
> seen -- the resulting group of nomcom participants is not random enough to
> ensure community comfort.

When the nomcom member volunteer pool was around 40, as it was for the
first several years, this problem didn't occur. It's only recently with
the larger pool and the phenomena of, relatively speaking, huge numbers
of volunteers from the same organization that this problem has shown up.  
One possibility is to make the criterion for volunteers more stringent
to get back down to a smaller number of volunteers. This would decrease
the number from any particular organization. And frankly, if three
people from a company ended up on the nomcom but they were all well
known, decades long participants in the IETF, it would not be much of a
problem. It's when some company routinely sends 45 people to IETF
meetings most of whom are relativley unknown to the community and 15 of
them volunteer and qualify for nomcom and 3+ of those are selected that
its a problem.

> The special nature of nomcom requires that we be proactive and prevent
> statistical anomalies, such as too many from the same company.
> 
> We need to consider 3 things:
> 
> 1.  What are the list of attributes that we need to worry about?
> 
>     Company affiliation is the one that has surfaced. Are there others?

It's not "company", it's organization. I don't think there are any 
others.

>     Would we be concerned if 8 nomcom members were from Malaysia, or from
>     the UK, or from China or from the US? What about having 8 being from
>     what, in the US, is called a "protected minority"? What about 8 being
>     mathematicians, or being psychologists? (Don't laugh. There are more of
>     the latter in the IETF than folks realize. As one of them, I am sure
>     that we should worry about it...) What if 8 had never been an editor,
>     working group chair, AD or IAB member -- ie, what if we had essentially
>     no voting member of nomcom with any real hands-on experience dealing
>     with the management process?

I just don't see much technical bias on the national level. Of course,
if you are talking government employees (which I assume you are not).  
then it would be a problemm, because they would be affiliated with the
same organization.

> 2.  What can we do to make randomization better, in terms of the pool that
>     members are drawn from and in terms of the mechanism for drawing from
>     that pool?

The randomization, if you use something like RFC 2777, is about as
random as you can get. Of course, its actualy a fairly powerful
mechanism and it would be trivial to bias it. For example, you could
make the probability of choosing volunter X be proportional to the
number of IETF meetings they have attended out of the past three plus 3
points for being a WG chair currently or within the past year plus 1
point up to a maximum of 3 for each non-obsolete standards track or BCP
RFC they have authored. You could make this as complex as you want, but
I'm not sure we need that yet or that it would be worth the endless
debates setting up the equation.

> 3.  What should be the hard limits, in case the randomization does not work?

Randomization works by being random. Thus it has a finite probability of
choosing any 10 from the list. If there is any such 10 whose selection
(say, because 3+ all work for the same company) is unacceptable and of
significant probability, then you need a hard limit. What is significant
probability? My opinion is something like 1 in 10**8 but people will
differ on this. However adopting a hard limit will set the probability
of unacceptable results, whatever the limit rule defines them to be, to
zero.

> Here are my own suggestions, just to offer a starting point for discussion:

I'm not sure we need any more discussion and I certainly don't think we
need to look for brand new areas at this time.

> 1.  Attributes of concern
> 
>     Company
>     Geography
>     IETF management experience.

From the point of view of an technology unbiased nomcom, both in reality
and appearance, organization (company) is the only concern.

"IETF management experience" is a whole different matter we should not 
confuse with the issue at hand.

> 2.  Randomization
> 
>     Use limits to protect against too many from company or geography. Make
>     separate selection pools from based on experience.
> 
>     Draw from the "experienced" pool first, up to 50% of the nomcom. Fold
>     the remainder of that pool into the second, pool and select the
>     remaining 50%.

There is no need for geography limits.

If you are going to define "experienced", rather than some complex
multistage selection process, I would just use that definition to bias
the random selection as I described above.

> 3.  No more than 3 from the same company.

Three is clearly too many. I would favor a limit of one from the same
organization but a limit of 2 will fix the problem and I support that as 
the consensus.

>     Use a standard table of geographic region and limit participation to no
>     more than 50% from a single region.

What standard? This is a political thicket it would be stupid to get
into since there isn't a problem here.

> d/
> 
> ps. This topic quickly gets distasteful. There are some attributes we would
> not want to acknowledge are a concern, even if he concern is valid. As a
> counterbalance, against this discomfort, we really do need to ask how the
> community will feel if nomcom has a very large number of participants with
> one of these "interesting" attributes. If the community would be worried
> when 8 out of the 10 voting nomcom members had a particular attribute in
> common, then we MUST find a way to mitigate against its occurrence.

I don't think so. If an attribute is so mild it takes 8 out of 10 or
before anyone is concerned, it seems implausible that it is worth
worrying about at this point. If you have a nomcom pool of 100 with 10
having a characteristic, the probability of choosing 3+ with that
characteristic is about 7% (we had a pool with multiple companies having
have more than 10 candidates in the pool resulting in a substantially
higher chance that at least one company would have 3+). The probabiliyt
of choosing 4+ was 1.3%. The probability of choosing 8+ under these
circumstances is only 9*10**-9. Presumably if 80 out of the 100 had that
characteristics, then 8 should be chosen.

Actually here is the table:

0	3.49E-01
1	3.87E-01
2	1.94E-01
3	5.74E-02
4	1.12E-02
5	1.49E-03
6	1.38E-04
7	8.75E-06
8	3.65E-07
9	9.00E-09
10	1.00E-10

> Sunday, November 10, 2002, 3:33:18 AM, you wrote:
> avri> - While the policy of the IETF is that individuals participate
> avri>    as individuals and not representatives of their companies,
> avri>    it is also accepted that in practice, many participants do
> avri>    represent their employer's viewpoints.
> ...
> avri> - If we do something to consider companies as participants in
> avri>    this instance, the IETF may need to do something about it
> avri>    in a broader sense.
> ...
> avri> -- However, since the nomcom process is not an open process,
> avri>     the normal safeguards against collaboration may not operate.

It can be more subtle than that. N nomcom members from the same 
organizatgion probably know the same people who are likely also 
affilitated with that organization and might be biased towards picking 
or avoiding such people, to the detriment of the IETF.

> d/

Thanks,
Donald
======================================================================
 Donald E. Eastlake 3rd                       dee3@torque.pothole.com
 155 Beaver Street              +1-508-634-2066(h) +1-508-851-8280(w)
 Milford, MA 01757 USA                   Donald.Eastlake@motorola.com