Knock Out refers to the player disabling an opponent by knocking them out with non-lethal weapons or with CQC.
Tranquillizer weapons can be used to put enemies to sleep. Often they will require multiple hits and the weapons have to be reloaded manually. Because of this, there is a possibility of them attacking the player after getting hit. Opponents can be stunned using stun grenades, using a choke or by hitting them enough times (a hit in the head or crotch = instant knockout). This method of disabling the player's opponents is encouraged, as they often give more items when searched if they have been knocked out over being killed.
In both situations given enough time they will wake up and cause an alert. If they are seen by another soldier they will kick them until they wake up. In Metal Gear Solid: Portable Ops, recruiting an opponent requires the player to knock them out before getting them back to the truck.
Stamina kill is an informal term to address defeating an opponent in battle by depleting their stamina. During boss battles, the life gauge of the player's opponent is displayed on screen, and oftentimes, the stamina gauge will be shown beneath it. Using stun weapons, the stamina gauge of the player's opponent can be reduced. Once it runs out they are knocked out. By using this method, the player will often receive special items for not "killing" the boss.
In the case of Portable Ops, the boss characters generally also require pre-determined goals to unlock them in addition to staminal killing them, with the exception of Python and Null, where the former doesn't have any requirements besides a stamina kill, and the latter becomes automatically unlockable for a second playthrough regardless of whether he was defeated with a lethal kill or a stamina kill. Metal Gear Solid: Peace Walker employs similar mechanic, allowing the player to non-lethally neutralize enemies and vehicles during boss battles, and fulton them afterwards.