主题：葫芦僧乱判葫芦案 -- 煮酒正熟

我就不信，这么基础的统计学知识，我还会搞错？？

个人觉得合理的办法是你在A组中愿意用卡的3k用户里面，只给1.5k用户发卡，然后对照这两个1.5k用户的结果。申请卡都有被拒可能，所以只发50%也不会有什么问题。

t 检验的原理之一就是比较两样本的均数是否一致。前提是样本来自总体并同质（符合总体分布，通常大样本都可以认为是正态分布，但这里不一样）。所以我觉得统计思想大有问题。再有，不能说同质就同质了，一定要有统计分析为依据。嗬嗬，统计认真起来，蛮有意思的，不是一个p 小于0.05 就搞定的。

In theory, what you've suggested is fine. But in reality, it's just not correct.

By turning down those 1,500 best customers of their right to use PLCC, you'll piss them off and hurt their loyalty to your brand. A certain percentage of these customers would switch to your competing brands and that's the least thing that you wanna see.

Secondly, from a pure theoretical stand point, the very act of denying the cards to those 1,500 people actually changes the quality of these people. To be more specific, this will hurt their loyalty, emotional attachment, and the passion associated with your brand, hence turn them to a lower quality group. In other words this group is no longer the group before! And you're still using them as the control group?? In this case you would over-estimate the impact of the card.

So the only theoretically correct and practically viable and appropriate approach is, you first randomly pick 5,000, propose the card to them, and let them decide at their free will. Then you'll have two groups of customers, the first group, say, 3000 people, being those who choose to use the card, and the second group (2,000) who choose not to use the card. These 3000 plus 2000 would be your Test Group.

Then you pick another 3000 people who belong to the same decile but are mutually exclusive from those 5000. This will be your control group.

Then you evaluate the performance of the whole Test Group (not only those 3000 card users) and the Control Group, do the comparisons and then draw a conclusion. This is exactly what we're doing right now, and the difference is more than 26%.

I don't see any flaws here. Can you?

No one would use t-statistic or F-statistic to draw a conclusion without checking the underlying assumptions.

If I remember correctly, the underlying assumptions include (but may not be limited to):

1. Normality;

2. Equal variance (or stable variance);

3. Linearity (no curvature)

4. no influential outliers

I didn't check the Statistics text book so I may miss one or two.

Assumptions checking can be done by the method of residual analysis.

What I'm trying to say here is that, these are the very basic knowledge in statistics. I do not hold any degree in statistics, but I still know that I need to check these assumptions before drawing a conclusion. I just can't imagine that my colleague in the Advanced Analytics team who holds PhD in mathematics and statistics would NOT know this. I think you might have been too subjective in reaching a conclusion that what our company does is not correct.

你总是说你们的统计博士不会犯这样的错。我也不认为他们在制定protocol时 会在这么简单的问题上有任何错误。问题是统计操作人员在选取样本的时候不知道他们在做什么，也不知道为什么这样做，对统计后面的思想和适用范围并不了然。我对spss和sas比较熟，见过初学统计的人在上面犯的钱其百怪的错误。

在我看来，你们这个统计过程的错误是很明显的， 你们作报告的时候比较的根本不是同一个总体（在你上一个帖子讲的方法开始往正确的方向行走了，但还有点问题。 你也解释了取样如何困难等等。然而作为受众， 我们关心的是你的统计过程和统计结论。取样的困难不意味着可以不恰当的延伸你的结论。统计是一份材料一分话，任何的延伸只会削弱结论的可信度）。我觉得老是引证统计博士如何如何没什么意思。你考过gmat,应该知道这对加强你的结论没有任何意义。这样的统计知识，国内本科就应该掌握，国外，是研究生的初级课程。不过一般人若不经常用，在此范迷糊可能也不奇怪。

我看了你昨晚的最后一个贴子。一些讲法，印证了我的看法，你对t 检验，齐性检验的原理理解似乎没有到位.本贴里，你又说 Assumptions checking can be done by the method of residual analysis.。 更是不知从何说起。

我本来是在短信和你探讨的，公开讨论非我所愿。我没有任何不敬的意思。如我短信所言，美国很多资深的医生在这里也常犯错误。据我所知，这不妨碍他们是很好的医生。

若有唐突，一笑置之可也。

I was not mad at you, or anyone. Not at all.

I'm also using SAS everyday, but not on statistical analysis. Our advanced analytics team folks also use SAS, and I believe the way that they check for nomality, equal variance, linearity, etc. is through running residual analysis. They plot the residuals and observe the distribution by eyes as a preliminary check (QQ plot for checking linearity, for example). Then they would also refer to more 'scientific' criteria such as a bunch of statistics (W for checking normality assumption). I don't remember all those statistics. Learned that at school but forget most of the stuff. But with a statistics textbook I can easily show you the statistics to check for those assumptions.

You keep saying that we were not using the comparables as Test and Control, but you never really explain why. You just keep saying "not comparable" like that. But why not comparable? Man, you've got to tell us more, and better yet, show us your design of experiment so that we could see clearly the difference between yours and our company's, and what advantages yours has over ours. Can you do me this favor?

It's really not something special, it's pretty much the same stuff that people in the whole industry are doing. If you challenge one specific analyst in our company that's fine. But since this is something really common in our industry, you're actually challenging all the statistics experts, and that' where I find a bit unaccetable.

By the way, just did a quick check and found that Levene test is for checking the equal variance assumption.

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35a.htm

The macro for levene test.

/*************************************************************************/

/* This macro tests the Equality of Variances for a Response Variable */

/* using the Levene Test and is available for use on the ASU's */

/* Research/Statistics UNIX cluster. */

/* */

/* The following SAS macro takes as input: */

/* - the name of a Classification or Grouping Variable */

/* - the name of the Dependent Variable to be analyzed */

/* - the name of the SAS Data Set containing the above two variables */

/* */

/* It uses SAS PROC GLM to perform Levene's Test for Homoscedasticity. */

/* */

/* References: */

/* Madansky, Albert, Prescriptions for Working Statisticians, 1988, */

/* New York: Springer-Verlag. */

/* Miller, Rupert G., Jr., Beyond ANOVA, Basics of Applied Statistics,*/

/* 1986, New York: Wiley. */

/* */

/* Example of a statement to call the macro below: */

/* %LEVENE(gp_varname,dp_varname,ds_name); */

/* where */

/* gp_varname is the name of the grouping variable to be tested, */

/* dp_varname is the name of the dependent variable to be tested */

/* for homoscedasticity, */

/* ds_name is the SAS data set name given in the DATA step. */

/* */

/* The MACRO can be called in your SAS program as indicated below: */

/* */

/* OPTIONS SAS_system_options; */

/* DATA ds_name; */

/* INPUT gp_varname, dp_varname; */

/* (rest of your SAS statements) */

/* . */

/* . */

/* RUN; */

/* TITLE "Your choice of title"; */

/* %LEVENE(gp_varname,dp_varname,ds_name); */

/*************************************************************************/

OPTIONS NONOTES NOSOURCE NOSOURCE2; /* suppress usual echoing, and */;

OPTIONS ERRORABEND; /* abort job if the macro fails */;

%Macro Levene(Gp_var,Dp_var,DSet);

Proc GLM Data=&DSet;

Class &Gp_var;

Model &Dp_var=&Gp_var;

Means &Gp_var/HOVTest;

ODS select HOVFTest;

Run;

Quit;

%Mend Levene;

t 检验的原理之一是样本均数和总体均数得比较。你报告时的两个组样本明显不是来自同一个整体。

大样本时通常无需正态检验，然而你们的样本特殊，可能要做。同时也还要做方差齐性检验。这些都不是residual analysis 能干的。可以分别在t 检验前 做Shapiro-Wilk 或 Kolmogorov-Smirnov test，和Levene's test,或 Bartlett's test。具体原理要画图讲。你可依找本教科书看。如果方差不齐，你们就只能作秩和检验了，统计结果也当然大打折扣。

强调的是，如果样本都是来自同一总体，鉴于你的样本是如此庞大，以上检验都不必做。这就是我说得你们的protocol 本来没错的原因。不幸的是，似乎你们在取样的时候没有考虑到这种特殊情况。

Why do you say our two groups are not from the same universe? I already explained several times, it's not the 3000 card users against the other 3000. It's the 5000 (among which there're 3000 card users) against the other 3000. Please tell me why they're not from the same "整体"?

As for assumptions validation, I believe you can do either way. You could first check the assumptions before running the test, or you could first run the test and then check for assumptions. In the first approach, if the assumptions do not hold, you need to recreate your testing samples, while in the second approach, if the assumptions do not hold, the quality / accuracy of your conclusion is at risk. It all depends on how serious those assumptions are violated, and which assumption is violated. (Normally the equal variance is more critical than normality.)

I don't know exactly how our statisticians are handling above problems, but I'm pretty sure they will exercise their professionalism in due course. How do you know they didn't do their job well? My original article didn't say our statisticians already checked for those assumptions, but this by no means should be interpretated as they did not check.

Finally, that VP does not know the first thing about statistics. So by pointing out any errors related to statistical analysis in our job does not prove or support the correctness / appropriateness of his decision of not accepting our proposal. If the VP were a statistician, then your challenge on our statistical analysis would make more sense.

我不是在挑战什么专家。我只是在阐述一个最基本的统计原理罢了。跟随原理出发，我觉得不会错在哪里去。我们可能和你们不太一样。每次作报告，尖锐地批评到处都是，我们都是很正面地看待这些的。不会认为这是对整个行业的挑战。如我再三强调，统计只有一个，即使你们有行业特殊性，但对不同总体的样本进行比较，又没有任何的同质均一性的分析，那我只能说你们的行业太特殊了。

PROTOCOL 本身没错，只不过在执行理解过程中会出偏差。

另外诚恳地说一句，齐性检验的方法有很多种。你举的方法只是其一。弄一段程序出来不能帮助你理解后面的原理。最好的方法还是弄懂其中的原理，这样就一通百通了，也会搞明白各种方法的优缺点以及为什么这样做。

我的意见你可以不接受，可能我说的却有不对的地方，以后我们慢慢提高吧。我对我们肺科医生的类似统计评论只用了两分钟，他们的PI现在看见我老远就打招呼。我在这儿打字其实也不图什么，西西河就是开心的地方。你若因为我的评论而加深了对你们统计方法的了解，那就再好不过了，即使我说的是错的。

只要你ＨＡＰＰＹ，我骂那个ＶＰ是猪都可以。只是没什么用。不能让他和你们做生意。你这个对我有意见，觉得你过了。

你们作报告的时候，是３０００卡人对３０００有意持卡人和可能无意持卡人。现在这个方案是后来的。怎么说呢，信誉有点影响吧。

这里你跟我强辩了吧。我猜你们的ｐｒｏｔｏｃｏｌ根本没这一条，因为正常情况下是不需要做的，原因上贴已讲。如果做了，凭我从你的原始描述和我的统计经验来看，不齐是肯定的。

我是从旁观者的角度来想的，人家做到这一级别，从销售起家的，特别是经常和同业一起聚会，只怕对这个什么卡的作用心里还是有一点认识的。应该有什么其它的东西影响她对你们这次数据挖掘的看法。

题外话，数据挖掘很热，我们医学也用，成功的不少，闹得笑话也不少。

我接小孩去了，咱就不聊了

So I may not be able to discuss statistics in detail. But I have general trust in our folks in the Advanced Analytics team. I have no problem at all with your challenging on statistical analysis method. The thing is, you don't yet know the bolts and nuts in our industry and our company, it's very likely that your challenge is just not valid. I have no doubt that you can always come up with great insights in your own research area, with the combination of an excellent knowledge about your subject and a solid understanding in statistics. Yet without the specific knowledge in another industry and research area and equipped only with statistics knowledge, it's likely to fall when making a challenge or judgment.