The yellow content of the upper layer replaces the yellow content of the lower layer composite. This is the "Y" in CMY and CMYK.
The display of the layer view presents only yellow information as a cue that the layer is in this layer mode. All of the red, green, blue, cyan, magenta, yellow, black, mask and inverted mask modes do this.
Where the higher layer is partially transparent, the result is mixed with the lower layer composite in a manner proportional to the amount of transparency.