wiki:ShuffleVector

Version 8 (modified by cameron, 4 years ago) (diff)

--

The ShuffleVector Project

This project investigates code generation for the LLVM shufflevector operation, particularly in the case that the shuffle mask is a compile-time constant.

For example, the shuffle mask pattern for a shuffle vector could be just a byte swap, for example.

%v3 = shufflevector <8 x i8> %v1, <8 x i8> undef,
                    <8 x i32> <i32 6, i32 7, i32 4, i32 5, i32 2, i32 3, i32 0, i32 1>  ; yields <8 x i8>

Transforming this to %t0 = bitcast %v1 to i64 @llvm.bswap.i64(i64 %t0) may allow efficient implementation on an architecture supporting byte swap, but not shuffle.

Generalizing this pattern, we may have arbitrary rotations expressed using shuffle masks. For example, consider the shufflevector of 4-bit fields:

%v3 = shufflevector <8 x i4> %v1, <8 x i4> undef,
              <8 x i32> <i32 1, i32 2, i32 3, i32 0, i32 5, i32 6, i32 7, i32 4>  ; yields <8 x i8>

Shuffles on 4-bit fields are generally not supported by SIMD instruction sets, but this one can be implemented by transforming to 16-bit vector shift operations.

%t0 = bitcast %v1 to <2 x i16>
%t1 = shl %t0, <2 x i16> <i16 12, i16 12> 
%t2 = lshr %t0, <2 x i16> <i16 4, i16 4> 
%v3 = xor %t1, %t2

Can these examples be turned into general rules that systematically capture these special cases?

Vectorized Sequential Code

If there is no known fully parallel implementation of a particular case, it may still be possible to partially parallelize by making a vectorized sequential loop.

Shuffling Bit-By-Bit

There are many interesting possibilities with shuffle masks that perform bit-by-bit selection. See the BitShuffle subproject.

LLVM Source Code and Shuffles

The following grep results show shuffle occurring throughout the LLVM code base in all kinds of special cases.

include/llvm-c/Core.h:  LLVMShuffleVector  = 52,
include/llvm-c/Core.h:      macro(ShuffleVectorInst)              \
include/llvm-c/Core.h:LLVMValueRef LLVMConstShuffleVector(LLVMValueRef VectorAConstant,
include/llvm-c/Core.h:LLVMValueRef LLVMBuildShuffleVector(LLVMBuilderRef, LLVMValueRef V1,
lib/Analysis/ConstantFolding.cpp:  case Instruction::ShuffleVector:
lib/Analysis/ConstantFolding.cpp:    return ConstantExpr::getShuffleVector(Ops[0], Ops[1], Ops[2]);
lib/Analysis/CostModel.cpp:static bool matchPairwiseShuffleMask(ShuffleVectorInst *SI, bool IsLeft,
lib/Analysis/CostModel.cpp:  // We don't need a shuffle if we just want to have element 0 in position 0 of
lib/Analysis/CostModel.cpp:  SmallVector<int, 16> ActualMask = SI->getShuffleMask();
lib/Analysis/CostModel.cpp:  // %rdx.shuf.0.0 = shufflevector <4 x float> %rdx, <4 x float> undef,
lib/Analysis/CostModel.cpp:  // %rdx.shuf.0.1 = shufflevector <4 x float> %rdx, <4 x float> undef,
lib/Analysis/CostModel.cpp:  ShuffleVectorInst *LS = dyn_cast<ShuffleVectorInst>(L);
lib/Analysis/CostModel.cpp:  ShuffleVectorInst *RS = dyn_cast<ShuffleVectorInst>(R);
lib/Analysis/CostModel.cpp:  // On level 0 we can omit one shufflevector instruction.
lib/Analysis/CostModel.cpp:  // Shuffle inputs must match.
lib/Analysis/CostModel.cpp:    // If we have two shuffles their operands must match.
lib/Analysis/CostModel.cpp:    // On the first level we can omit the shufflevector <0, undef,...>. So the
lib/Analysis/CostModel.cpp:    // input to the other shufflevector <1, undef> must match with one of the
lib/Analysis/CostModel.cpp:    //  %NextLevelOpL = shufflevector %R, <1, undef ...>
lib/Analysis/CostModel.cpp:  // Shuffle mask for pairwise operation must match.
lib/Analysis/CostModel.cpp:  if (matchPairwiseShuffleMask(LS, true, Level)) {
lib/Analysis/CostModel.cpp:    if (!matchPairwiseShuffleMask(RS, false, Level))
lib/Analysis/CostModel.cpp:  } else if (matchPairwiseShuffleMask(RS, true, Level)) {
lib/Analysis/CostModel.cpp:    if (!matchPairwiseShuffleMask(LS, false, Level))
lib/Analysis/CostModel.cpp:  // We look for a sequence of shuffle,shuffle,add triples like the following
lib/Analysis/CostModel.cpp:  // %rdx.shuf.0.0 = shufflevector <4 x float> %rdx, <4 x float> undef,
lib/Analysis/CostModel.cpp:  // %rdx.shuf.0.1 = shufflevector <4 x float> %rdx, <4 x float> undef,
lib/Analysis/CostModel.cpp:  // %rdx.shuf.1.0 = shufflevector <4 x float> %bin.rdx.0, <4 x float> undef,
lib/Analysis/CostModel.cpp:  // %rdx.shuf.1.1 = shufflevector <4 x float> %bin.rdx.0, <4 x float> undef,
lib/Analysis/CostModel.cpp:static std::pair<Value *, ShuffleVectorInst *>
lib/Analysis/CostModel.cpp:getShuffleAndOtherOprd(BinaryOperator *B) {
lib/Analysis/CostModel.cpp:  ShuffleVectorInst *S = 0;
lib/Analysis/CostModel.cpp:  if ((S = dyn_cast<ShuffleVectorInst>(L)))
lib/Analysis/CostModel.cpp:  S = dyn_cast<ShuffleVectorInst>(R);
lib/Analysis/CostModel.cpp:  // We look for a sequence of shuffles and adds like the following matching one
lib/Analysis/CostModel.cpp:  // fadd, shuffle vector pair at a time.
lib/Analysis/CostModel.cpp:  // %rdx.shuf = shufflevector <4 x float> %rdx, <4 x float> undef,
lib/Analysis/CostModel.cpp:  // %rdx.shuf7 = shufflevector <4 x float> %bin.rdx, <4 x float> undef,
lib/Analysis/CostModel.cpp:  SmallVector<int, 32> ShuffleMask(NumVecElems, 0);
lib/Analysis/CostModel.cpp:    ShuffleVectorInst *Shuffle;
lib/Analysis/CostModel.cpp:    std::tie(NextRdxOp, Shuffle) = getShuffleAndOtherOprd(BinOp);
lib/Analysis/CostModel.cpp:    // Check the current reduction operation and the shuffle use the same value.
lib/Analysis/CostModel.cpp:    if (Shuffle == 0)
lib/Analysis/CostModel.cpp:    if (Shuffle->getOperand(0) != NextRdxOp)
lib/Analysis/CostModel.cpp:    // Check that shuffle masks matches.
lib/Analysis/CostModel.cpp:      ShuffleMask[j] = MaskStart + j;
lib/Analysis/CostModel.cpp:    std::fill(&ShuffleMask[MaskStart], ShuffleMask.end(), -1);
lib/Analysis/CostModel.cpp:    SmallVector<int, 16> Mask = Shuffle->getShuffleMask();
lib/Analysis/CostModel.cpp:    if (!matchMask(ShuffleMask, Mask))
lib/Analysis/CostModel.cpp:    // Try to match a reduction sequence (series of shufflevector and vector
lib/Analysis/CostModel.cpp:  case Instruction::ShuffleVector: {
lib/Analysis/CostModel.cpp:    const ShuffleVectorInst *Shuffle = cast<ShuffleVectorInst>(I);
lib/Analysis/CostModel.cpp:    Type *VecTypOp0 = Shuffle->getOperand(0)->getType();
lib/Analysis/CostModel.cpp:    SmallVector<int, 16> Mask = Shuffle->getShuffleMask();
lib/Analysis/CostModel.cpp:      return TTI->getShuffleCost(TargetTransformInfo::SK_Reverse, VecTypOp0, 0,
lib/Analysis/Lint.cpp:  // TODO: Look through vector insert/extract/shuffle.
lib/Analysis/TargetTransformInfo.cpp:unsigned TargetTransformInfo::getShuffleCost(ShuffleKind Kind, Type *Tp,
lib/Analysis/TargetTransformInfo.cpp:  return PrevTTI->getShuffleCost(Kind, Tp, Index, SubTp);
lib/Analysis/TargetTransformInfo.cpp:  unsigned getShuffleCost(ShuffleKind Kind, Type *Ty,
lib/AsmParser/LLLexer.cpp:  INSTKEYWORD(shufflevector,  ShuffleVector);
lib/AsmParser/LLParser.cpp:  case lltok::kw_shufflevector:
lib/AsmParser/LLParser.cpp:    } else if (Opc == Instruction::ShuffleVector) {
lib/AsmParser/LLParser.cpp:        return Error(ID.Loc, "expected three operands to shufflevector");
lib/AsmParser/LLParser.cpp:      if (!ShuffleVectorInst::isValidOperands(Elts[0], Elts[1], Elts[2]))
lib/AsmParser/LLParser.cpp:        return Error(ID.Loc, "invalid operands to shufflevector");
lib/AsmParser/LLParser.cpp:                 ConstantExpr::getShuffleVector(Elts[0], Elts[1],Elts[2]);
lib/AsmParser/LLParser.cpp:  case lltok::kw_shufflevector:  return ParseShuffleVector(Inst, PFS);
lib/AsmParser/LLParser.cpp:/// ParseShuffleVector
lib/AsmParser/LLParser.cpp:///   ::= 'shufflevector' TypeAndValue ',' TypeAndValue ',' TypeAndValue
lib/AsmParser/LLParser.cpp:bool LLParser::ParseShuffleVector(Instruction *&Inst, PerFunctionState &PFS) {
lib/AsmParser/LLParser.cpp:      ParseToken(lltok::comma, "expected ',' after shuffle mask") ||
lib/AsmParser/LLParser.cpp:      ParseToken(lltok::comma, "expected ',' after shuffle value") ||
lib/AsmParser/LLParser.cpp:  if (!ShuffleVectorInst::isValidOperands(Op0, Op1, Op2))
lib/AsmParser/LLParser.cpp:    return Error(Loc, "invalid shufflevector operands");
lib/AsmParser/LLParser.cpp:  Inst = new ShuffleVectorInst(Op0, Op1, Op2);
lib/AsmParser/LLParser.h:    bool ParseShuffleVector(Instruction *&I, PerFunctionState &PFS);
lib/AsmParser/LLToken.h:    kw_extractelement, kw_insertelement, kw_shufflevector,
lib/CodeGen/BasicTargetTransformInfo.cpp:  virtual unsigned getShuffleCost(ShuffleKind Kind, Type *Tp,
lib/CodeGen/BasicTargetTransformInfo.cpp:unsigned BasicTTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
lib/CodeGen/BasicTargetTransformInfo.cpp:  // Assume the pairwise shuffles add a cost.
lib/CodeGen/BasicTargetTransformInfo.cpp:  unsigned ShuffleCost =
lib/CodeGen/BasicTargetTransformInfo.cpp:      TopTTI->getShuffleCost(SK_ExtractSubvector, Ty, NumVecElts / 2, Ty);
lib/CodeGen/BasicTargetTransformInfo.cpp:  return ShuffleCost + ArithCost + getScalarizationOverhead(Ty, false, true);
lib/CodeGen/CodeGenPrepare.cpp:    bool OptimizeShuffleVectorInst(ShuffleVectorInst *SI);
lib/CodeGen/CodeGenPrepare.cpp:static bool isBroadcastShuffle(ShuffleVectorInst *SVI) {
lib/CodeGen/CodeGenPrepare.cpp:  SmallVector<int, 16> Mask(SVI->getShuffleMask());
lib/CodeGen/CodeGenPrepare.cpp:/// it's often worth sinking a shufflevector splat down to its use so that
lib/CodeGen/CodeGenPrepare.cpp:bool CodeGenPrepare::OptimizeShuffleVectorInst(ShuffleVectorInst *SVI) {
lib/CodeGen/CodeGenPrepare.cpp:  // We only expect better codegen by sinking a shuffle if we can recognise a
lib/CodeGen/CodeGenPrepare.cpp:  if (!isBroadcastShuffle(SVI))
lib/CodeGen/CodeGenPrepare.cpp:  // InsertedShuffles - Only insert a shuffle in each block once.
lib/CodeGen/CodeGenPrepare.cpp:  DenseMap<BasicBlock*, Instruction*> InsertedShuffles;
lib/CodeGen/CodeGenPrepare.cpp:    // Everything checks out, sink the shuffle if the user's block doesn't
lib/CodeGen/CodeGenPrepare.cpp:    Instruction *&InsertedShuffle = InsertedShuffles[UserBB];
lib/CodeGen/CodeGenPrepare.cpp:    if (!InsertedShuffle) {
lib/CodeGen/CodeGenPrepare.cpp:      InsertedShuffle = new ShuffleVectorInst(SVI->getOperand(0),
lib/CodeGen/CodeGenPrepare.cpp:    User->replaceUsesOfWith(SVI, InsertedShuffle);
lib/CodeGen/CodeGenPrepare.cpp:  // If we removed all uses, nuke the shuffle.
lib/CodeGen/CodeGenPrepare.cpp:  if (ShuffleVectorInst *SVI = dyn_cast<ShuffleVectorInst>(I))
lib/CodeGen/CodeGenPrepare.cpp:    return OptimizeShuffleVectorInst(SVI);
lib/CodeGen/MachineScheduler.cpp:// Machine Instruction Shuffler for Correctness Testing
lib/CodeGen/MachineScheduler.cpp:class InstructionShuffler : public MachineSchedStrategy {
lib/CodeGen/MachineScheduler.cpp:  InstructionShuffler(bool alternate, bool topdown)
lib/CodeGen/MachineScheduler.cpp:static ScheduleDAGInstrs *createInstructionShuffler(MachineSchedContext *C) {
lib/CodeGen/MachineScheduler.cpp:  return new ScheduleDAGMILive(C, new InstructionShuffler(Alternate, TopDown));
lib/CodeGen/MachineScheduler.cpp:static MachineSchedRegistry ShufflerRegistry(
lib/CodeGen/MachineScheduler.cpp:  "shuffle", "Shuffle machine instructions alternating directions",
lib/CodeGen/MachineScheduler.cpp:  createInstructionShuffler);
lib/CodeGen/TargetLoweringBase.cpp:  case ShuffleVector:  return ISD::VECTOR_SHUFFLE;
lib/IR/AsmWriter.cpp:    // Select, Store and ShuffleVector always print all types.
lib/IR/AsmWriter.cpp:    if (isa<SelectInst>(I) || isa<StoreInst>(I) || isa<ShuffleVectorInst>(I)
lib/IR/AutoUpgrade.cpp:        Rep = Builder.CreateShuffleVector(Op0, Op0, ConstantVector::get(Idxs));
lib/IR/ConstantFold.cpp:Constant *llvm::ConstantFoldShuffleVectorInstruction(Constant *V1,
lib/IR/ConstantFold.cpp:  // Undefined shuffle mask -> undefined value.
lib/IR/ConstantFold.cpp:  // Loop over the shuffle mask, evaluating each element.
lib/IR/ConstantFold.cpp:    int Elt = ShuffleVectorInst::getMaskValue(Mask, i);
lib/IR/ConstantFold.h:  Constant *ConstantFoldShuffleVectorInstruction(Constant *V1, Constant *V2,
lib/IR/Constants.cpp:  case Instruction::ShuffleVector:
lib/IR/Constants.cpp:    return ConstantExpr::getShuffleVector(Ops[0], Ops[1], Ops[2]);
lib/IR/Constants.cpp:Constant *ConstantExpr::getShuffleVector(Constant *V1, Constant *V2, 
lib/IR/Constants.cpp:  assert(ShuffleVectorInst::isValidOperands(V1, V2, Mask) &&
lib/IR/Constants.cpp:         "Invalid shuffle vector constant expr operands!");
lib/IR/Constants.cpp:  if (Constant *FC = ConstantFoldShuffleVectorInstruction(V1, V2, Mask))
lib/IR/Constants.cpp:  const ExprMapKeyType Key(Instruction::ShuffleVector, ArgVec);
lib/IR/Constants.cpp:  case Instruction::ShuffleVector:
lib/IR/Constants.cpp:    return new ShuffleVectorInst(Ops[0], Ops[1], Ops[2]);
lib/IR/ConstantsContext.h:/// ShuffleVectorConstantExpr - This class is private to
lib/IR/ConstantsContext.h:/// shufflevector constant exprs.
lib/IR/ConstantsContext.h:class ShuffleVectorConstantExpr : public ConstantExpr {
lib/IR/ConstantsContext.h:  ShuffleVectorConstantExpr(Constant *C1, Constant *C2, Constant *C3)
lib/IR/ConstantsContext.h:                 Instruction::ShuffleVector, 
lib/IR/ConstantsContext.h:struct OperandTraits<ShuffleVectorConstantExpr> :
lib/IR/ConstantsContext.h:    public FixedNumOperandTraits<ShuffleVectorConstantExpr, 3> {
lib/IR/ConstantsContext.h:DEFINE_TRANSPARENT_OPERAND_ACCESSORS(ShuffleVectorConstantExpr, Value)
lib/IR/ConstantsContext.h:    if (V.opcode == Instruction::ShuffleVector)
lib/IR/ConstantsContext.h:      return new ShuffleVectorConstantExpr(V.operands[0], V.operands[1],
lib/IR/Core.cpp:LLVMValueRef LLVMConstShuffleVector(LLVMValueRef VectorAConstant,
lib/IR/Core.cpp:  return wrap(ConstantExpr::getShuffleVector(unwrap<Constant>(VectorAConstant),
lib/IR/Core.cpp:LLVMValueRef LLVMBuildShuffleVector(LLVMBuilderRef B, LLVMValueRef V1,
lib/IR/Core.cpp:  return wrap(unwrap(B)->CreateShuffleVector(unwrap(V1), unwrap(V2),
lib/IR/Instruction.cpp:  case ShuffleVector:  return "shufflevector";
lib/IR/Instructions.cpp://                      ShuffleVectorInst Implementation
lib/IR/Instructions.cpp:ShuffleVectorInst::ShuffleVectorInst(Value *V1, Value *V2, Value *Mask,
lib/IR/Instructions.cpp:              ShuffleVector,
lib/IR/Instructions.cpp:              OperandTraits<ShuffleVectorInst>::op_begin(this),
lib/IR/Instructions.cpp:              OperandTraits<ShuffleVectorInst>::operands(this),
lib/IR/Instructions.cpp:         "Invalid shuffle vector instruction operands!");
lib/IR/Instructions.cpp:ShuffleVectorInst::ShuffleVectorInst(Value *V1, Value *V2, Value *Mask,
lib/IR/Instructions.cpp:              ShuffleVector,
lib/IR/Instructions.cpp:              OperandTraits<ShuffleVectorInst>::op_begin(this),
lib/IR/Instructions.cpp:              OperandTraits<ShuffleVectorInst>::operands(this),
lib/IR/Instructions.cpp:         "Invalid shuffle vector instruction operands!");
lib/IR/Instructions.cpp:bool ShuffleVectorInst::isValidOperands(const Value *V1, const Value *V2,
lib/IR/Instructions.cpp:  // used as the shuffle mask. When this occurs, the shuffle mask will
lib/IR/Instructions.cpp:/// getMaskValue - Return the index from the shuffle mask for the specified
lib/IR/Instructions.cpp:int ShuffleVectorInst::getMaskValue(Constant *Mask, unsigned i) {
lib/IR/Instructions.cpp:/// getShuffleMask - Return the full mask for this instruction, where each
lib/IR/Instructions.cpp:void ShuffleVectorInst::getShuffleMask(Constant *Mask,
lib/IR/Instructions.cpp:ShuffleVectorInst *ShuffleVectorInst::clone_impl() const {
lib/IR/Instructions.cpp:  return new ShuffleVectorInst(getOperand(0), getOperand(1), getOperand(2));
lib/IR/LLVMContextImpl.cpp:void ShuffleVectorConstantExpr::anchor() { }
lib/IR/Verifier.cpp:  void visitShuffleVectorInst(ShuffleVectorInst &EI);
lib/IR/Verifier.cpp:void Verifier::visitShuffleVectorInst(ShuffleVectorInst &SV) {
lib/IR/Verifier.cpp:  Assert1(ShuffleVectorInst::isValidOperands(SV.getOperand(0), SV.getOperand(1),
lib/IR/Verifier.cpp:          "Invalid shufflevector operands!", &SV);
Robs-MacBook-Pro:llvm cameron$ 
Robs-MacBook-Pro:llvm cameron$ fgrep huffle [li]*/*/*/*.*
include/llvm/ADT/IntervalMap.h:    assert(CurSize[n] == NewSize[n] && "Insufficient element shuffle");
include/llvm/Analysis/TargetFolder.h:  Constant *CreateShuffleVector(Constant *V1, Constant *V2,
include/llvm/Analysis/TargetFolder.h:    return Fold(ConstantExpr::getShuffleVector(V1, V2, Mask));
include/llvm/Analysis/TargetTransformInfo.h:  /// \brief The various kinds of shuffle patterns for vector queries.
include/llvm/Analysis/TargetTransformInfo.h:  enum ShuffleKind {
include/llvm/Analysis/TargetTransformInfo.h:  /// \return The cost of a shuffle instruction of kind Kind and of type Tp.
include/llvm/Analysis/TargetTransformInfo.h:  /// extraction shuffle kinds.
include/llvm/Analysis/TargetTransformInfo.h:  virtual unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index = 0,
include/llvm/CodeGen/ISDOpcodes.h:    /// ShuffleVectorSDNode class.  This is quite similar to the Altivec
include/llvm/CodeGen/SelectionDAG.h:  /// getVectorShuffle - Return an ISD::VECTOR_SHUFFLE node.  The number of
include/llvm/CodeGen/SelectionDAG.h:  SDValue getVectorShuffle(EVT VT, SDLoc dl, SDValue N1, SDValue N2,
include/llvm/CodeGen/SelectionDAGNodes.h:/// ShuffleVectorSDNode - This SDNode is used to implement the code generator
include/llvm/CodeGen/SelectionDAGNodes.h:/// support for the llvm IR shufflevector instruction.  It combines elements
include/llvm/CodeGen/SelectionDAGNodes.h:/// the shuffle mask.  For input vectors of width N, mask indices of 0..N-1
include/llvm/CodeGen/SelectionDAGNodes.h:class ShuffleVectorSDNode : public SDNode {
include/llvm/CodeGen/SelectionDAGNodes.h:  ShuffleVectorSDNode(EVT VT, unsigned Order, DebugLoc dl, SDValue N1,
include/llvm/IR/ConstantFolder.h:  Constant *CreateShuffleVector(Constant *V1, Constant *V2,
include/llvm/IR/ConstantFolder.h:    return ConstantExpr::getShuffleVector(V1, V2, Mask);
include/llvm/IR/Constants.h:  static Constant *getShuffleVector(Constant *V1, Constant *V2, Constant *Mask);
include/llvm/IR/IRBuilder.h:  Value *CreateShuffleVector(Value *V1, Value *V2, Value *Mask,
include/llvm/IR/IRBuilder.h:          return Insert(Folder.CreateShuffleVector(V1C, V2C, MC), Name);
include/llvm/IR/IRBuilder.h:    return Insert(new ShuffleVectorInst(V1, V2, Mask), Name);
include/llvm/IR/IRBuilder.h:    // First insert it into an undef vector so we can shuffle it.
include/llvm/IR/IRBuilder.h:    // Shuffle the value across the desired number of elements.
include/llvm/IR/IRBuilder.h:    return CreateShuffleVector(V, Undef, Zeros, Name + ".splat");
include/llvm/IR/InstVisitor.h:  RetTy visitShuffleVectorInst(ShuffleVectorInst &I) { DELEGATE(Instruction);}
include/llvm/IR/Instruction.def:HANDLE_OTHER_INST(56, ShuffleVector, ShuffleVectorInst)  // shuffle two vectors.
include/llvm/IR/Instructions.h://                           ShuffleVectorInst Class
include/llvm/IR/Instructions.h:/// ShuffleVectorInst - This instruction constructs a fixed permutation of two
include/llvm/IR/Instructions.h:class ShuffleVectorInst : public Instruction {
include/llvm/IR/Instructions.h:  ShuffleVectorInst *clone_impl() const override;
include/llvm/IR/Instructions.h:  ShuffleVectorInst(Value *V1, Value *V2, Value *Mask,
include/llvm/IR/Instructions.h:  ShuffleVectorInst(Value *V1, Value *V2, Value *Mask,
include/llvm/IR/Instructions.h:  /// isValidOperands - Return true if a shufflevector instruction can be
include/llvm/IR/Instructions.h:  /// getMaskValue - Return the index from the shuffle mask for the specified
include/llvm/IR/Instructions.h:  /// getShuffleMask - Return the full mask for this instruction, where each
include/llvm/IR/Instructions.h:  static void getShuffleMask(Constant *Mask, SmallVectorImpl<int> &Result);
include/llvm/IR/Instructions.h:  void getShuffleMask(SmallVectorImpl<int> &Result) const {
include/llvm/IR/Instructions.h:    return getShuffleMask(getMask(), Result);
include/llvm/IR/Instructions.h:  SmallVector<int, 16> getShuffleMask() const {
include/llvm/IR/Instructions.h:    getShuffleMask(Mask);
include/llvm/IR/Instructions.h:    return I->getOpcode() == Instruction::ShuffleVector;
include/llvm/IR/Instructions.h:struct OperandTraits<ShuffleVectorInst> :
include/llvm/IR/Instructions.h:  public FixedNumOperandTraits<ShuffleVectorInst, 3> {
include/llvm/IR/Instructions.h:DEFINE_TRANSPARENT_OPERAND_ACCESSORS(ShuffleVectorInst, Value)
include/llvm/IR/IntrinsicsPowerPC.td:  // vpkuhum is lowered to a shuffle.
include/llvm/IR/IntrinsicsPowerPC.td:  // vpkuwum is lowered to a shuffle.
include/llvm/IR/IntrinsicsX86.td:// Shuffle ops
include/llvm/IR/NoFolder.h:  Instruction *CreateShuffleVector(Constant *V1, Constant *V2,
include/llvm/IR/NoFolder.h:    return new ShuffleVectorInst(V1, V2, Mask);
include/llvm/Target/TargetLowering.h:  virtual bool isShuffleMaskLegal(const SmallVectorImpl<int> &/*Mask*/,
include/llvm/Target/TargetLowering.h:  /// Similar to isShuffleMaskLegal. This is used by Targets can use this to
include/llvm/Target/TargetSelectionDAG.td:def SDTVecShuffle : SDTypeProfile<1, 2, [
include/llvm/Target/TargetSelectionDAG.td:def vector_shuffle : SDNode<"ISD::VECTOR_SHUFFLE", SDTVecShuffle, []>;
lib/Bitcode/Reader/BitcodeReader.cpp:      V = ConstantExpr::getShuffleVector(Op0, Op1, Op2);
lib/Bitcode/Reader/BitcodeReader.cpp:      V = ConstantExpr::getShuffleVector(Op0, Op1, Op2);
lib/Bitcode/Reader/BitcodeReader.cpp:      I = new ShuffleVectorInst(Vec1, Vec2, Mask);
lib/Bitcode/Writer/BitcodeWriter.cpp:      case Instruction::ShuffleVector:
lib/Bitcode/Writer/BitcodeWriter.cpp:        // standard shufflevector instruction.  If the types are different,
lib/Bitcode/Writer/BitcodeWriter.cpp:        // then the shuffle is widening or truncating the input vectors, and
lib/Bitcode/Writer/BitcodeWriter.cpp:  case Instruction::ShuffleVector:
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    SDValue XformToShuffleWithZero(SDNode *N);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // Xor/and/or are indifferent to the swizzle operation (shuffle of one value).
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // If both shuffles use the same mask, and both shuffle within a single
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // vector types from memory. In many cases this allows additional shuffle
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    ShuffleVectorSDNode *SVN0 = cast<ShuffleVectorSDNode>(N0);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    ShuffleVectorSDNode *SVN1 = cast<ShuffleVectorSDNode>(N1);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:           "Inputs to shuffles are not the same type");
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    // Check that both shuffles use the same mask. The masks are known to be of
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:      return DAG.getVectorShuffle(VT, SDLoc(N), Op,
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(InVec);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // Perform only after legalization to ensure build_vector / vector_shuffle
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // (vextract (v4f32 shuffle (load $addr), <1,u,u,u>), 0) -> (f32 load $addr)
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    const ShuffleVectorSDNode *SVN = NULL;
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    } else if ((SVN = dyn_cast<ShuffleVectorSDNode>(InVec))) {
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:      // (vextract (vector_shuffle (load $addr), v2, <1, u, u, u>), 1)
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // using shuffles.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // turn into a single shuffle instruction.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // at most two distinct vectors, turn this into a shuffle node.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // May only combine to shuffle after legalize if shuffle is legal.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    // If everything is good, we can make a shuffle operation.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    // We can't generate a shuffle node with mismatched input and output types.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    // Only type-legal BUILD_VECTOR nodes are converted to shuffle nodes.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    return DAG.getVectorShuffle(VT, dl, Ops[0], Ops[1], &Mask[0]);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // inputs come from at most two distinct vectors, turn this into a shuffle
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:// Tries to turn a shuffle of two CONCAT_VECTORS into a single concat.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:static SDValue partitionShuffleOfConcats(SDNode *N, SelectionDAG &DAG) {
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  ShuffleVectorSDNode *SVN = cast<ShuffleVectorSDNode>(N);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  assert(N0.getValueType() == VT && "Vector shuffle must be normalized in DAG");
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // Canonicalize shuffle undef, undef -> undef
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  ShuffleVectorSDNode *SVN = cast<ShuffleVectorSDNode>(N);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // Canonicalize shuffle v, v -> v, undef
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    return DAG.getVectorShuffle(VT, SDLoc(N), N0, DAG.getUNDEF(VT),
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // Canonicalize shuffle undef, v -> v, undef.  Commute the shuffle mask.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    return DAG.getVectorShuffle(VT, SDLoc(N), N1, DAG.getUNDEF(VT),
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:      return DAG.getVectorShuffle(VT, SDLoc(N), N0, N1, &NewMask[0]);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    SDValue V = partitionShuffleOfConcats(N, DAG);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // If this shuffle node is simply a swizzle of another shuffle node,
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // and it reverses the swizzle of the previous shuffle then we can
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  // optimize shuffle(shuffle(x, undef), undef) -> x.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    ShuffleVectorSDNode *OtherSV = cast<ShuffleVectorSDNode>(N0);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    // Shuffle nodes can only reverse shuffles with a single non-undef value.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    // The incoming shuffle must be of the same type as the result of the
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:    // current shuffle.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:           "Shuffle types don't match");
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:      // shuffle. Adopt the incoming index.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:      // The combined shuffle must map each index to itself.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:/// XformToShuffleWithZero - Returns a vector_shuffle if it able to transform
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:/// an AND to a vector_shuffle with the destination vector and a zero vector.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:///      vector_shuffle V, Zero, <0, 4, 2, 4>
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:SDValue DAGCombiner::XformToShuffleWithZero(SDNode *N) {
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:      // Let's see if the target supports this vector_shuffle.
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:      SDValue Shuf = DAG.getVectorShuffle(RVT, dl, LHS, Zero, &Indices[0]);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  SDValue Shuffle = XformToShuffleWithZero(N);
lib/CodeGen/SelectionDAG/DAGCombiner.cpp:  if (Shuffle.getNode()) return Shuffle;
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:  /// ShuffleWithNarrowerEltType - Return a vector shuffle operation which
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:  /// whose vector element type is narrower than the original shuffle type.
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:  SDValue ShuffleWithNarrowerEltType(EVT NVT, EVT VT, SDLoc dl,
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:/// ShuffleWithNarrowerEltType - Return a vector shuffle operation which
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:/// whose vector element type is narrower than the original shuffle type.
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:SelectionDAGLegalize::ShuffleWithNarrowerEltType(EVT NVT, EVT VT,  SDLoc dl,
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:    return DAG.getVectorShuffle(NVT, dl, N1, N2, &Mask[0]);
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:  assert(TLI.isShuffleMaskLegal(NewMask, NVT) && "Shuffle not legal?");
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:  return DAG.getVectorShuffle(NVT, dl, N1, N2, &NewMask[0]);
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:      // We generate a shuffle of InVec and ScVec, so the shuffle mask
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:      return DAG.getVectorShuffle(Vec.getValueType(), dl, Vec, ScVec,
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:    SmallVector<int, 8> ShuffleVec(NumElems, -1);
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:      ShuffleVec[i] = V == Value1 ? 0 : NumElems;
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:    if (TLI.isShuffleMaskLegal(ShuffleVec, Node->getValueType(0))) {
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:      // Return shuffle(LowValVec, undef, <0,0,0,0>)
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:      return DAG.getVectorShuffle(VT, dl, Vec1, Vec2, ShuffleVec.data());
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:    ArrayRef<int> Mask = cast<ShuffleVectorSDNode>(Node)->getMask();
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:        // Convert shuffle node.
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:        // Convert the shuffle mask
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:    ArrayRef<int> Mask = cast<ShuffleVectorSDNode>(Node)->getMask();
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:    // Convert the shuffle mask to the right # elements.
lib/CodeGen/SelectionDAG/LegalizeDAG.cpp:    Tmp1 = ShuffleWithNarrowerEltType(NVT, OVT, dl, Tmp1, Tmp2, Mask);
lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp:  ShuffleVectorSDNode *SV = cast<ShuffleVectorSDNode>(N);
lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp:  return DAG.getVectorShuffle(OutVT, dl, V0, V1, &NewMask[0]);
lib/CodeGen/SelectionDAG/LegalizeTypes.h:  void SplitVecRes_VECTOR_SHUFFLE(ShuffleVectorSDNode *N, SDValue &Lo,
lib/CodeGen/SelectionDAG/LegalizeTypes.h:  SDValue WidenVecRes_VECTOR_SHUFFLE(ShuffleVectorSDNode *N);
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:    SplitVecRes_VECTOR_SHUFFLE(cast<ShuffleVectorSDNode>(N), Lo, Hi);
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:void DAGTypeLegalizer::SplitVecRes_VECTOR_SHUFFLE(ShuffleVectorSDNode *N,
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:  // express it as a vector shuffle of those two inputs.  Otherwise extract the
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:    // Build a shuffle mask for the output, discovering on the fly which
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:    // input vectors to use as shuffle operands (recorded in InputUsed).
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:    // If building a suitable shuffle vector proves too hard, then bail
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:      // Find or create a shuffle vector operand to hold this input.
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:        // shuffle vector.  Insert all elements into a BUILD_VECTOR instead.
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:      // Add the mask index for the new shuffle vector.
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:      // At least one input vector was used.  Create a new shuffle vector.
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:      Output =  DAG.getVectorShuffle(NewVT, dl, Op0, Op1, &Ops[0]);
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:    Res = WidenVecRes_VECTOR_SHUFFLE(cast<ShuffleVectorSDNode>(N));
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:        // Replace concat of two operands with a shuffle.
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:        return DAG.getVectorShuffle(WidenVT, dl,
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:SDValue DAGTypeLegalizer::WidenVecRes_VECTOR_SHUFFLE(ShuffleVectorSDNode *N) {
lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:  return DAG.getVectorShuffle(WidenVT, dl, InOp1, InOp2, &NewMask[0]);
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:    const ShuffleVectorSDNode *SVN = cast<ShuffleVectorSDNode>(N);
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  assert(!isa<ShuffleVectorSDNode>(N) && "Bad ShuffleVectorSDNode!");
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:    // vector shuffle in this situation). However, we do not need any code to
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:// commuteShuffle - swaps the values of N1 and N2, and swaps all indices in
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:// the shuffle mask M that point at N1 to point at N2, and indices that point
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:static void commuteShuffle(SDValue &N1, SDValue &N2, SmallVectorImpl<int> &M) {
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:SDValue SelectionDAG::getVectorShuffle(EVT VT, SDLoc dl, SDValue N1,
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // Canonicalize shuffle undef, undef -> undef
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // input to the shuffle.
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // Canonicalize shuffle v, v -> v, undef
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // Canonicalize shuffle undef, v -> v, undef.  Commute the shuffle mask.
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:    commuteShuffle(N1, N2, MaskVec);
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // Canonicalize all index into lhs, -> shuffle lhs, undef
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // Canonicalize all index into rhs, -> shuffle rhs, undef
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:    commuteShuffle(N1, N2, MaskVec);
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // If Identity shuffle return that node.
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  ShuffleVectorSDNode *N =
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:    new (NodeAllocator) ShuffleVectorSDNode(VT, dl.getIROrder(),
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:    llvm_unreachable("should use getVectorShuffle constructor!");
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:bool ShuffleVectorSDNode::isSplatMask(const int *Mask, EVT VT) {
lib/CodeGen/SelectionDAG/SelectionDAG.cpp:  // Find the first non-undef value in the shuffle mask.
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:      // Vector widening case, e.g. <2 x float> -> <4 x float>.  Shuffle in
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:// Utility for visitShuffleVector - Return true if every element in Mask,
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:void SelectionDAGBuilder::visitShuffleVector(const User &I) {
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:  ShuffleVectorInst::getShuffleMask(cast<Constant>(I.getOperand(2)), Mask);
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:    setValue(&I, DAG.getVectorShuffle(VT, getCurSDLoc(), Src1, Src2,
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:  // Normalize the shuffle vector since mask and vector length don't match.
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:        // The shuffle is concatenating two vectors together.
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:        // The shuffle is concatenating two vectors together.
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:    setValue(&I, DAG.getVectorShuffle(VT, getCurSDLoc(), Src1, Src2,
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:    // two subvectors and do the shuffle. The analysis is done by calculating
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:      // Extract appropriate subvector and generate a vector shuffle
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:      setValue(&I, DAG.getVectorShuffle(VT, getCurSDLoc(), Src1, Src2,
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp:  // replacing the shuffle with extract and build vector.
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h:class ShuffleVectorInst;
lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h:  void visitShuffleVector(const User &I);
lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp:  case ISD::VECTOR_SHUFFLE:             return "vector_shuffle";
lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp:  } else if (const ShuffleVectorSDNode *SVN =
lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp:               dyn_cast<ShuffleVectorSDNode>(this)) {
lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:STATISTIC(NumFastIselFailShuffleVector,"Fast isel fails on ShuffleVector");
lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:  case Instruction::ShuffleVector:  NumFastIselFailShuffleVector++; return;
lib/ExecutionEngine/Interpreter/Execution.cpp:void Interpreter::visitShuffleVectorInst(ShuffleVectorInst &I){
lib/ExecutionEngine/Interpreter/Execution.cpp:    llvm_unreachable("Unhandled dest type for shufflevector instruction");
lib/ExecutionEngine/Interpreter/Execution.cpp:  // shufflevector instruction.
lib/ExecutionEngine/Interpreter/Execution.cpp:          // %tmp = shufflevector <2 x i32> <i32 3, i32 4>, <2 x i32> undef,
lib/ExecutionEngine/Interpreter/Execution.cpp:          llvm_unreachable("Invalid mask in shufflevector instruction");
lib/ExecutionEngine/Interpreter/Execution.cpp:          llvm_unreachable("Invalid mask in shufflevector instruction");
lib/ExecutionEngine/Interpreter/Execution.cpp:          llvm_unreachable("Invalid mask in shufflevector instruction");
lib/ExecutionEngine/Interpreter/Interpreter.h:  void visitShuffleVectorInst(ShuffleVectorInst &I);
lib/Target/AArch64/AArch64ISelLowering.cpp:  SDValue ShuffleVec
lib/Target/AArch64/AArch64ISelLowering.cpp:    = DAG.getVectorShuffle(DestVT, SDLoc(N), Sext, DAG.getUNDEF(DestVT), Mask);
lib/Target/AArch64/AArch64ISelLowering.cpp:                SubRegVT, ShuffleVec,
lib/Target/AArch64/AArch64ISelLowering.cpp:// Check whether a shuffle_vector could be presented as concat_vector.
lib/Target/AArch64/AArch64ISelLowering.cpp:// Check whether a Build Vector could be presented as Shuffle Vector.
lib/Target/AArch64/AArch64ISelLowering.cpp:// This Shuffle Vector maybe not legalized, so the length of its operand and
lib/Target/AArch64/AArch64ISelLowering.cpp:bool AArch64TargetLowering::isKnownShuffleVector(SDValue Op, SelectionDAG &DAG,
lib/Target/AArch64/AArch64ISelLowering.cpp:  // Try to lower this in lowering ShuffleVector way.
lib/Target/AArch64/AArch64ISelLowering.cpp:  if (isKnownShuffleVector(Op, DAG, V0, V1, Mask)) {
lib/Target/AArch64/AArch64ISelLowering.cpp:      SDValue Shuffle = DAG.getVectorShuffle(VT, DL, V0, V1, Mask);
lib/Target/AArch64/AArch64ISelLowering.cpp:      if (Shuffle.getOpcode() != ISD::VECTOR_SHUFFLE)
lib/Target/AArch64/AArch64ISelLowering.cpp:        return Shuffle;
lib/Target/AArch64/AArch64ISelLowering.cpp:        return LowerVECTOR_SHUFFLE(Shuffle, DAG);
lib/Target/AArch64/AArch64ISelLowering.cpp:  // scalar_to_vector for the elements followed by a shuffle (provided the
lib/Target/AArch64/AArch64ISelLowering.cpp:  // shuffle is valid for the target) and materialization element by element
lib/Target/AArch64/AArch64ISelLowering.cpp:/// isREVMask - Check if a vector shuffle corresponds to a REV
lib/Target/AArch64/AArch64ISelLowering.cpp:  // If the first shuffle index is UNDEF, be optimistic.
lib/Target/AArch64/AArch64ISelLowering.cpp:// isPermuteMask - Check whether the vector shuffle matches to UZP, ZIP and
lib/Target/AArch64/AArch64ISelLowering.cpp:  ShuffleVectorSDNode *SVN = cast<ShuffleVectorSDNode>(Op.getNode());
lib/Target/AArch64/AArch64ISelLowering.cpp:  // Convert shuffles that are directly supported on NEON to target-specific
lib/Target/AArch64/AArch64ISelLowering.cpp:  // DAG nodes, instead of keeping them as shuffles and matching them again
lib/Target/AArch64/AArch64ISelLowering.cpp:  ArrayRef<int> ShuffleMask = SVN->getMask();
lib/Target/AArch64/AArch64ISelLowering.cpp:  if (isREVMask(ShuffleMask, VT, 64))
lib/Target/AArch64/AArch64ISelLowering.cpp:  if (isREVMask(ShuffleMask, VT, 32))
lib/Target/AArch64/AArch64ISelLowering.cpp:  if (isREVMask(ShuffleMask, VT, 16))
lib/Target/AArch64/AArch64ISelLowering.cpp:    ISDNo = isPermuteMask(ShuffleMask, VT, true);
lib/Target/AArch64/AArch64ISelLowering.cpp:    ISDNo = isPermuteMask(ShuffleMask, VT, false);
lib/Target/AArch64/AArch64ISelLowering.cpp:  if (isConcatVector(Op, DAG, V1, V2, &ShuffleMask[0], Res))
lib/Target/AArch64/AArch64ISelLowering.cpp:  // If the element of shuffle mask are all the same constant, we can
lib/Target/AArch64/AArch64ISelLowering.cpp:  if (ShuffleVectorSDNode::isSplatMask(&ShuffleMask[0], VT)) {
lib/Target/AArch64/AArch64ISelLowering.cpp:  int Length = ShuffleMask.size();
lib/Target/AArch64/AArch64ISelLowering.cpp:  // If the number of v1 elements is the same as the number of shuffle mask
lib/Target/AArch64/AArch64ISelLowering.cpp:  // element and the shuffle masks are sequential values, we can transform
lib/Target/AArch64/AArch64ISelLowering.cpp:    // Check if the shuffle mask is sequential.
lib/Target/AArch64/AArch64ISelLowering.cpp:    while (ShuffleMask[SkipUndef] == -1) {
lib/Target/AArch64/AArch64ISelLowering.cpp:    int CurMask = ShuffleMask[SkipUndef];
lib/Target/AArch64/AArch64ISelLowering.cpp:        if (ShuffleMask[I] != -1 && ShuffleMask[I] != CurMask) {
lib/Target/AArch64/AArch64ISelLowering.cpp:        unsigned Index = (EltSize / 8) * (ShuffleMask[SkipUndef] - SkipUndef);
lib/Target/AArch64/AArch64ISelLowering.cpp:  // For shuffle mask like "0, 1, 2, 3, 4, 5, 13, 7", try to generate insert
lib/Target/AArch64/AArch64ISelLowering.cpp:  // If shuffle mask is like "0, 1, 10, 11, 12, 13, 14, 15", V2 would be a
lib/Target/AArch64/AArch64ISelLowering.cpp:    if (ShuffleMask[I] != I) {
lib/Target/AArch64/AArch64ISelLowering.cpp:      NV1Elt.push_back(ShuffleMask[I]);
lib/Target/AArch64/AArch64ISelLowering.cpp:    if (ShuffleMask[I] != (I + V1EltNum)) {
lib/Target/AArch64/AArch64ISelLowering.cpp:      NV2Elt.push_back(ShuffleMask[I]);
lib/Target/AArch64/AArch64ISelLowering.h:  bool isKnownShuffleVector(SDValue Op, SelectionDAG &DAG, SDValue &V0,
lib/Target/ARM/ARMISelLowering.cpp:#include "ARMPerfectShuffle.h"
lib/Target/ARM/ARMISelLowering.cpp:// check if an VEXT instruction can handle the shuffle mask when the
lib/Target/ARM/ARMISelLowering.cpp:// vector sources of the shuffle are the same.
lib/Target/ARM/ARMISelLowering.cpp:  // Assume that the first shuffle index is not UNDEF.  Fail if it is.
lib/Target/ARM/ARMISelLowering.cpp:  // If this is a VEXT shuffle, the immediate value is the index of the first
lib/Target/ARM/ARMISelLowering.cpp:  // element.  The other shuffle indices must be the successive elements after
lib/Target/ARM/ARMISelLowering.cpp:  // Assume that the first shuffle index is not UNDEF.  Fail if it is.
lib/Target/ARM/ARMISelLowering.cpp:  // If this is a VEXT shuffle, the immediate value is the index of the first
lib/Target/ARM/ARMISelLowering.cpp:  // element.  The other shuffle indices must be the successive elements after
lib/Target/ARM/ARMISelLowering.cpp:/// isVREVMask - Check if a vector shuffle corresponds to a VREV
lib/Target/ARM/ARMISelLowering.cpp:  // If the first shuffle index is UNDEF, be optimistic.
lib/Target/ARM/ARMISelLowering.cpp:  // We can handle <8 x i8> vector shuffles. If the index in the mask is out of
lib/Target/ARM/ARMISelLowering.cpp:/// "vector_shuffle v, v", i.e., "vector_shuffle v, undef".
lib/Target/ARM/ARMISelLowering.cpp:/// "vector_shuffle v, v", i.e., "vector_shuffle v, undef".
lib/Target/ARM/ARMISelLowering.cpp:/// "vector_shuffle v, v", i.e., "vector_shuffle v, undef".
lib/Target/ARM/ARMISelLowering.cpp:    SDValue shuffle = ReconstructShuffle(Op, DAG);
lib/Target/ARM/ARMISelLowering.cpp:    if (shuffle != SDValue())
lib/Target/ARM/ARMISelLowering.cpp:      return shuffle;
lib/Target/ARM/ARMISelLowering.cpp:  // scalar_to_vector for the elements followed by a shuffle (provided the
lib/Target/ARM/ARMISelLowering.cpp:  // shuffle is valid for the target) and materialization element by element
lib/Target/ARM/ARMISelLowering.cpp:// shuffle in combination with VEXTs.
lib/Target/ARM/ARMISelLowering.cpp:SDValue ARMTargetLowering::ReconstructShuffle(SDValue Op,
lib/Target/ARM/ARMISelLowering.cpp:      // A shuffle can only come from building a vector from various
lib/Target/ARM/ARMISelLowering.cpp:      // This code doesn't know how to handle shuffles where the vector
lib/Target/ARM/ARMISelLowering.cpp:  SDValue ShuffleSrcs[2] = {DAG.getUNDEF(VT), DAG.getUNDEF(VT) };
lib/Target/ARM/ARMISelLowering.cpp:  // and prepares appropriate SDValues for a shuffle if possible.
lib/Target/ARM/ARMISelLowering.cpp:      ShuffleSrcs[i] = SourceVecs[i];
lib/Target/ARM/ARMISelLowering.cpp:      // break it down again in a shuffle.
lib/Target/ARM/ARMISelLowering.cpp:           "unexpected vector sizes in ReconstructShuffle");
lib/Target/ARM/ARMISelLowering.cpp:      ShuffleSrcs[i] = DAG.getNode(ISD::EXTRACT_SUBVECTOR, dl, VT,
lib/Target/ARM/ARMISelLowering.cpp:      ShuffleSrcs[i] = DAG.getNode(ISD::EXTRACT_SUBVECTOR, dl, VT,
lib/Target/ARM/ARMISelLowering.cpp:      ShuffleSrcs[i] = DAG.getNode(ARMISD::VEXT, dl, VT, VEXTSrc1, VEXTSrc2,
lib/Target/ARM/ARMISelLowering.cpp:  if (isShuffleMaskLegal(Mask, VT))
lib/Target/ARM/ARMISelLowering.cpp:    return DAG.getVectorShuffle(VT, dl, ShuffleSrcs[0], ShuffleSrcs[1],
lib/Target/ARM/ARMISelLowering.cpp:/// isShuffleMaskLegal - Targets can use this to indicate that they only
lib/Target/ARM/ARMISelLowering.cpp:ARMTargetLowering::isShuffleMaskLegal(const SmallVectorImpl<int> &M,
lib/Target/ARM/ARMISelLowering.cpp:    // Compute the index in the perfect shuffle table.
lib/Target/ARM/ARMISelLowering.cpp:    unsigned PFEntry = PerfectShuffleTable[PFTableIndex];
lib/Target/ARM/ARMISelLowering.cpp:          ShuffleVectorSDNode::isSplatMask(&M[0], VT) ||
lib/Target/ARM/ARMISelLowering.cpp:/// GeneratePerfectShuffle - Given an entry in the perfect-shuffle table, emit
lib/Target/ARM/ARMISelLowering.cpp:/// the specified operations to build the shuffle.
lib/Target/ARM/ARMISelLowering.cpp:static SDValue GeneratePerfectShuffle(unsigned PFEntry, SDValue LHS,
lib/Target/ARM/ARMISelLowering.cpp:  OpLHS = GeneratePerfectShuffle(PerfectShuffleTable[LHSID], LHS, RHS, DAG, dl);
lib/Target/ARM/ARMISelLowering.cpp:  OpRHS = GeneratePerfectShuffle(PerfectShuffleTable[RHSID], LHS, RHS, DAG, dl);
lib/Target/ARM/ARMISelLowering.cpp:  default: llvm_unreachable("Unknown shuffle opcode!");
lib/Target/ARM/ARMISelLowering.cpp:                                       ArrayRef<int> ShuffleMask,
lib/Target/ARM/ARMISelLowering.cpp:         I = ShuffleMask.begin(), E = ShuffleMask.end(); I != E; ++I)
lib/Target/ARM/ARMISelLowering.cpp:  ShuffleVectorSDNode *SVN = cast<ShuffleVectorSDNode>(Op.getNode());
lib/Target/ARM/ARMISelLowering.cpp:  // Convert shuffles that are directly supported on NEON to target-specific
lib/Target/ARM/ARMISelLowering.cpp:  // DAG nodes, instead of keeping them as shuffles and matching them again
lib/Target/ARM/ARMISelLowering.cpp:  ArrayRef<int> ShuffleMask = SVN->getMask();
lib/Target/ARM/ARMISelLowering.cpp:    if (ShuffleVectorSDNode::isSplatMask(&ShuffleMask[0], VT)) {
lib/Target/ARM/ARMISelLowering.cpp:    if (isVEXTMask(ShuffleMask, VT, ReverseVEXT, Imm)) {
lib/Target/ARM/ARMISelLowering.cpp:    if (isVREVMask(ShuffleMask, VT, 64))
lib/Target/ARM/ARMISelLowering.cpp:    if (isVREVMask(ShuffleMask, VT, 32))
lib/Target/ARM/ARMISelLowering.cpp:    if (isVREVMask(ShuffleMask, VT, 16))
lib/Target/ARM/ARMISelLowering.cpp:        isSingletonVEXTMask(ShuffleMask, VT, Imm)) {
lib/Target/ARM/ARMISelLowering.cpp:    // Check for Neon shuffles that modify both input vectors in place.
lib/Target/ARM/ARMISelLowering.cpp:    // If both results are used, i.e., if there are two shuffles with the same
lib/Target/ARM/ARMISelLowering.cpp:    // used for both shuffles.
lib/Target/ARM/ARMISelLowering.cpp:    if (isVTRNMask(ShuffleMask, VT, WhichResult))
lib/Target/ARM/ARMISelLowering.cpp:    if (isVUZPMask(ShuffleMask, VT, WhichResult))
lib/Target/ARM/ARMISelLowering.cpp:    if (isVZIPMask(ShuffleMask, VT, WhichResult))
lib/Target/ARM/ARMISelLowering.cpp:    if (isVTRN_v_undef_Mask(ShuffleMask, VT, WhichResult))
lib/Target/ARM/ARMISelLowering.cpp:    if (isVUZP_v_undef_Mask(ShuffleMask, VT, WhichResult))
lib/Target/ARM/ARMISelLowering.cpp:    if (isVZIP_v_undef_Mask(ShuffleMask, VT, WhichResult))
lib/Target/ARM/ARMISelLowering.cpp:  // If the shuffle is not directly supported and it has 4 elements, use
lib/Target/ARM/ARMISelLowering.cpp:  // the PerfectShuffle-generated table to synthesize it from other shuffles.
lib/Target/ARM/ARMISelLowering.cpp:      if (ShuffleMask[i] < 0)
lib/Target/ARM/ARMISelLowering.cpp:        PFIndexes[i] = ShuffleMask[i];
lib/Target/ARM/ARMISelLowering.cpp:    // Compute the index in the perfect shuffle table.
lib/Target/ARM/ARMISelLowering.cpp:    unsigned PFEntry = PerfectShuffleTable[PFTableIndex];
lib/Target/ARM/ARMISelLowering.cpp:      return GeneratePerfectShuffle(PFEntry, V1, V2, DAG, dl);
lib/Target/ARM/ARMISelLowering.cpp:  // Implement shuffles with 32- or 64-bit elements as ARMISD::BUILD_VECTORs.
lib/Target/ARM/ARMISelLowering.cpp:      if (ShuffleMask[i] < 0)
lib/Target/ARM/ARMISelLowering.cpp:                                  ShuffleMask[i] < (int)NumElts ? V1 : V2,
lib/Target/ARM/ARMISelLowering.cpp:                                  DAG.getConstant(ShuffleMask[i] & (NumElts-1),
lib/Target/ARM/ARMISelLowering.cpp:  if ((VT == MVT::v8i16 || VT == MVT::v16i8) && isReverseMask(ShuffleMask, VT))
lib/Target/ARM/ARMISelLowering.cpp:    SDValue NewOp = LowerVECTOR_SHUFFLEv8i8(Op, ShuffleMask, DAG);
lib/Target/ARM/ARMISelLowering.cpp:  // Optimize trunc store (of multiple scalars) to shuffle and store.  First,
lib/Target/ARM/ARMISelLowering.cpp:    // Create a type on which we perform the shuffle.
lib/Target/ARM/ARMISelLowering.cpp:    SmallVector<int, 8> ShuffleVec(NumElems * SizeRatio, -1);
lib/Target/ARM/ARMISelLowering.cpp:    for (unsigned i = 0; i < NumElems; ++i) ShuffleVec[i] = i * SizeRatio;
lib/Target/ARM/ARMISelLowering.cpp:    // Can't shuffle using an illegal type.
lib/Target/ARM/ARMISelLowering.cpp:    SDValue Shuff = DAG.getVectorShuffle(WideVecVT, DL, WideVec,
lib/Target/ARM/ARMISelLowering.cpp:                                ShuffleVec.data());
lib/Target/ARM/ARMISelLowering.cpp:  // The LLVM shufflevector instruction does not require the shuffle mask
lib/Target/ARM/ARMISelLowering.cpp:  //   shuffle(concat(v1, undef), concat(v2, undef)) ->
lib/Target/ARM/ARMISelLowering.cpp:  //   shuffle(concat(v1, v2), undef)
lib/Target/ARM/ARMISelLowering.cpp:  // Translate the shuffle mask.
lib/Target/ARM/ARMISelLowering.cpp:  ShuffleVectorSDNode *SVN = cast<ShuffleVectorSDNode>(N);
lib/Target/ARM/ARMISelLowering.cpp:  return DAG.getVectorShuffle(VT, SDLoc(N), NewConcat,
lib/Target/ARM/ARMISelLowering.h:      // Vector shuffles:
lib/Target/ARM/ARMISelLowering.h:      VTBL1,        // 1-register shuffle with mask
lib/Target/ARM/ARMISelLowering.h:      VTBL2,        // 2-register shuffle with mask
lib/Target/ARM/ARMISelLowering.h:      // is fine if BUILD_VECTORs are always lowered to shuffles or other
lib/Target/ARM/ARMISelLowering.h:    bool isShuffleMaskLegal(const SmallVectorImpl<int> &M, EVT VT) const;
lib/Target/ARM/ARMISelLowering.h:    SDValue ReconstructShuffle(SDValue Op, SelectionDAG &DAG) const;
lib/Target/ARM/ARMInstrNEON.td:// 2-register shuffles (VTRN/VZIP/VUZP), both double- and quad-register.
lib/Target/ARM/ARMInstrNEON.td:class N2VDShuffle<bits<2> op19_18, bits<5> op11_7, string OpcodeStr, string Dt>
lib/Target/ARM/ARMInstrNEON.td:class N2VQShuffle<bits<2> op19_18, bits<5> op11_7,
lib/Target/ARM/ARMInstrNEON.td:// Other Vector Shuffles.
lib/Target/ARM/ARMInstrNEON.td:def  VTRNd8   : N2VDShuffle<0b00, 0b00001, "vtrn", "8">;
lib/Target/ARM/ARMInstrNEON.td:def  VTRNd16  : N2VDShuffle<0b01, 0b00001, "vtrn", "16">;
lib/Target/ARM/ARMInstrNEON.td:def  VTRNd32  : N2VDShuffle<0b10, 0b00001, "vtrn", "32">;
lib/Target/ARM/ARMInstrNEON.td:def  VTRNq8   : N2VQShuffle<0b00, 0b00001, IIC_VPERMQ, "vtrn", "8">;
lib/Target/ARM/ARMInstrNEON.td:def  VTRNq16  : N2VQShuffle<0b01, 0b00001, IIC_VPERMQ, "vtrn", "16">;
lib/Target/ARM/ARMInstrNEON.td:def  VTRNq32  : N2VQShuffle<0b10, 0b00001, IIC_VPERMQ, "vtrn", "32">;
lib/Target/ARM/ARMInstrNEON.td:def  VUZPd8   : N2VDShuffle<0b00, 0b00010, "vuzp", "8">;
lib/Target/ARM/ARMInstrNEON.td:def  VUZPd16  : N2VDShuffle<0b01, 0b00010, "vuzp", "16">;
lib/Target/ARM/ARMInstrNEON.td:def  VUZPq8   : N2VQShuffle<0b00, 0b00010, IIC_VPERMQ3, "vuzp", "8">;
lib/Target/ARM/ARMInstrNEON.td:def  VUZPq16  : N2VQShuffle<0b01, 0b00010, IIC_VPERMQ3, "vuzp", "16">;
lib/Target/ARM/ARMInstrNEON.td:def  VUZPq32  : N2VQShuffle<0b10, 0b00010, IIC_VPERMQ3, "vuzp", "32">;
lib/Target/ARM/ARMInstrNEON.td:def  VZIPd8   : N2VDShuffle<0b00, 0b00011, "vzip", "8">;
lib/Target/ARM/ARMInstrNEON.td:def  VZIPd16  : N2VDShuffle<0b01, 0b00011, "vzip", "16">;
lib/Target/ARM/ARMInstrNEON.td:def  VZIPq8   : N2VQShuffle<0b00, 0b00011, IIC_VPERMQ3, "vzip", "8">;
lib/Target/ARM/ARMInstrNEON.td:def  VZIPq16  : N2VQShuffle<0b01, 0b00011, IIC_VPERMQ3, "vzip", "16">;
lib/Target/ARM/ARMInstrNEON.td:def  VZIPq32  : N2VQShuffle<0b10, 0b00011, IIC_VPERMQ3, "vzip", "32">;
lib/Target/ARM/ARMPerfectShuffle.h://===-- ARMPerfectShuffle.h - NEON Perfect Shuffle Table --------*- C++ -*-===//
lib/Target/ARM/ARMPerfectShuffle.h:// This file, which was autogenerated by llvm-PerfectShuffle, contains data
lib/Target/ARM/ARMPerfectShuffle.h:// for the optimal way to build a perfect shuffle using neon instructions.
lib/Target/ARM/ARMPerfectShuffle.h:static const unsigned PerfectShuffleTable[6561+1] = {
lib/Target/ARM/ARMTargetTransformInfo.cpp:  unsigned getShuffleCost(ShuffleKind Kind, Type *Tp,
lib/Target/ARM/ARMTargetTransformInfo.cpp:unsigned ARMTTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
lib/Target/ARM/ARMTargetTransformInfo.cpp:  // We only handle costs of reverse shuffles for now.
lib/Target/ARM/ARMTargetTransformInfo.cpp:    return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp);
lib/Target/ARM/ARMTargetTransformInfo.cpp:  static const CostTblEntry<MVT::SimpleValueType> NEONShuffleTbl[] = {
lib/Target/ARM/ARMTargetTransformInfo.cpp:    // Reverse shuffle cost one instruction if we are shuffling within a double
lib/Target/ARM/ARMTargetTransformInfo.cpp:    // word (vrev) or two if we shuffle a quad word (vrev, vext).
lib/Target/ARM/ARMTargetTransformInfo.cpp:  int Idx = CostTableLookup(NEONShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second);
lib/Target/ARM/ARMTargetTransformInfo.cpp:    return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp);
lib/Target/ARM/ARMTargetTransformInfo.cpp:  return LT.first * NEONShuffleTbl[Idx].Cost;
lib/Target/CppBackend/CPPBackend.cpp:      case Instruction::ShuffleVector:  Out << "getShuffleVector("; break;
lib/Target/CppBackend/CPPBackend.cpp:  case Instruction::ShuffleVector: {
lib/Target/CppBackend/CPPBackend.cpp:    const ShuffleVectorInst* svi = cast<ShuffleVectorInst>(I);
lib/Target/CppBackend/CPPBackend.cpp:    Out << "ShuffleVectorInst* " << getCppName(svi)
lib/Target/CppBackend/CPPBackend.cpp:        << " = new ShuffleVectorInst(" << opNames[0]
lib/Target/Hexagon/HexagonIntrinsics.td:// STYPE / PERM / Vector shuffle.
lib/Target/Mips/MSA.txt:        It is not possible to emit vshf.w when the shuffle description is
lib/Target/Mips/MSA.txt:        instead. It is also impossible for the shuffle description to be
lib/Target/Mips/MSA.txt:        unknown at compile-time due to the definition of shufflevector in
lib/Target/Mips/MSA.txt:        When the shuffle description describes a splat operation, splat.[bhwd]
lib/Target/Mips/MSA.txt:        same shuffle. ilvev.d will be emitted instead.
lib/Target/Mips/MSA.txt:        same shuffle. ilvod.d will be emitted instead.
lib/Target/Mips/MipsISelLowering.h:      // Vector Shuffle with mask as an operand
lib/Target/Mips/MipsISelLowering.h:      VSHF,  // Generic shuffle
lib/Target/Mips/MipsISelLowering.h:      SHF,   // 4-element set shuffle.
lib/Target/Mips/MipsMSAInstrInfo.td:// Big endian bitcasts expand to shuffle instructions.
lib/Target/Mips/MipsSEISelDAGToDAG.cpp://       sometimes a shuffle in big-endian mode.
lib/Target/Mips/MipsSEISelDAGToDAG.cpp://       sometimes a shuffle in big-endian mode.
lib/Target/Mips/MipsSEISelDAGToDAG.cpp://       sometimes a shuffle in big-endian mode.
lib/Target/Mips/MipsSEISelDAGToDAG.cpp://       sometimes a shuffle in big-endian mode.
lib/Target/Mips/MipsSEISelLowering.cpp:    // We can't lower via VECTOR_SHUFFLE because it requires constant shuffle
lib/Target/Mips/MipsSEISelLowering.cpp:// SHF splits the vector into blocks of four elements, then shuffles these
lib/Target/Mips/MipsSEISelLowering.cpp://   %2 = shufflevector <8 x i16> %0, <8 x i16> undef,
lib/Target/Mips/MipsSEISelLowering.cpp:// This mostly consists of converting the shuffle indices in Indices into a
lib/Target/Mips/MipsSEISelLowering.cpp:    llvm_unreachable("shuffle vector mask references neither vector operand?");
lib/Target/Mips/MipsSEISelLowering.cpp:// indices in the shuffle.
lib/Target/Mips/MipsSEISelLowering.cpp:  ShuffleVectorSDNode *Node = cast<ShuffleVectorSDNode>(Op);
lib/Target/Mips/MipsSEISelLowering.h:    virtual bool isShuffleMaskLegal(const SmallVectorImpl<int> &Mask,
lib/Target/Mips/MipsSEISelLowering.h:    /// depending on the indices in the shuffle.
lib/Target/NVPTX/NVPTX.h:  VecShuffle = 4,
lib/Target/NVPTX/NVPTXGenericToNVVM.cpp:  case Instruction::ShuffleVector:
lib/Target/NVPTX/NVPTXGenericToNVVM.cpp:    // ShuffleVector
lib/Target/NVPTX/NVPTXGenericToNVVM.cpp:    return Builder.CreateShuffleVector(NewOperands[0], NewOperands[1],
lib/Target/NVPTX/NVPTXInstrInfo.td:def isVecShuffle : VecInstTypeEnum<4>;
lib/Target/NVPTX/NVPTXVector.td:class ShuffleOneLine<string vecsize, string elem, string type>
lib/Target/NVPTX/NVPTXVector.td:class ShuffleAsmStr2<string type>
lib/Target/NVPTX/NVPTXVector.td:  string t1 = ShuffleOneLine<"2", "0", type>.s;
lib/Target/NVPTX/NVPTXVector.td:  string s  = !strconcat(t2, ShuffleOneLine<"2", "1", type>.s);
lib/Target/NVPTX/NVPTXVector.td:class ShuffleAsmStr4<string type>
lib/Target/NVPTX/NVPTXVector.td:  string t1 = ShuffleOneLine<"4", "0", type>.s;
lib/Target/NVPTX/NVPTXVector.td:  string t3 = !strconcat(t2, ShuffleOneLine<"4", "1", type>.s);
lib/Target/NVPTX/NVPTXVector.td:  string t5 = !strconcat(t4, ShuffleOneLine<"4", "2", type>.s);
lib/Target/NVPTX/NVPTXVector.td:  string s  = !strconcat(t6, ShuffleOneLine<"4", "3", type>.s);
lib/Target/NVPTX/NVPTXVector.td:let neverHasSideEffects=1, VecInstType=isVecShuffle.Value in {
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v4f32 : NVPTXVecInst<(outs V4F32Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr4<"f32">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v4i32 : NVPTXVecInst<(outs V4I32Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr4<"u32">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v4i16 : NVPTXVecInst<(outs V4I16Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr4<"u16">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v4i8 : NVPTXVecInst<(outs V4I8Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr4<"u16">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v2f32 : NVPTXVecInst<(outs V2F32Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr2<"f32">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v2i32 : NVPTXVecInst<(outs V2I32Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr2<"u32">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v2i8 : NVPTXVecInst<(outs V2I8Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr2<"u16">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v2i16 : NVPTXVecInst<(outs V2I16Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr2<"u16">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v2f64 : NVPTXVecInst<(outs V2F64Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr2<"f64">.s),
lib/Target/NVPTX/NVPTXVector.td:def VecShuffle_v2i64 : NVPTXVecInst<(outs V2I64Regs:$dst),
lib/Target/NVPTX/NVPTXVector.td:                                 ShuffleAsmStr2<"u64">.s),
lib/Target/NVPTX/NVPTXVector.td:def ShuffleMask0 : SDNodeXForm<vector_shuffle, [{
lib/Target/NVPTX/NVPTXVector.td:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(N);
lib/Target/NVPTX/NVPTXVector.td:def ShuffleMask1 : SDNodeXForm<vector_shuffle, [{
lib/Target/NVPTX/NVPTXVector.td:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(N);
lib/Target/NVPTX/NVPTXVector.td:def ShuffleMask2 : SDNodeXForm<vector_shuffle, [{
lib/Target/NVPTX/NVPTXVector.td:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(N);
lib/Target/NVPTX/NVPTXVector.td:def ShuffleMask3 : SDNodeXForm<vector_shuffle, [{
lib/Target/NVPTX/NVPTXVector.td:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(N);
lib/Target/NVPTX/NVPTXVector.td:                       (vector_shuffle node:$lhs, node:$rhs),
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v2f64 V2F64Regs:$src1, V2F64Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v4f32 V4F32Regs:$src1, V4F32Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op),
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask2 node:$op), (ShuffleMask3 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v2f32 V2F32Regs:$src1, V2F32Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v2i64 V2I64Regs:$src1, V2I64Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v4i32 V4I32Regs:$src1, V4I32Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op),
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask2 node:$op), (ShuffleMask3 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v2i32 V2I32Regs:$src1, V2I32Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v4i16 V4I16Regs:$src1, V4I16Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op),
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask2 node:$op), (ShuffleMask3 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v2i16 V2I16Regs:$src1, V2I16Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v4i8 V4I8Regs:$src1, V4I8Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op),
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask2 node:$op), (ShuffleMask3 node:$op))>;
lib/Target/NVPTX/NVPTXVector.td:          (VecShuffle_v2i8 V2I8Regs:$src1, V2I8Regs:$src2,
lib/Target/NVPTX/NVPTXVector.td:                            (ShuffleMask0 node:$op), (ShuffleMask1 node:$op))>;
lib/Target/PowerPC/PPCISelLowering.cpp:#include "PPCPerfectShuffle.h"
lib/Target/PowerPC/PPCISelLowering.cpp:      // We promote all shuffles to v16i8.
lib/Target/PowerPC/PPCISelLowering.cpp:/// isVPKUHUMShuffleMask - Return true if this is the shuffle mask for a
lib/Target/PowerPC/PPCISelLowering.cpp:bool PPC::isVPKUHUMShuffleMask(ShuffleVectorSDNode *N, bool isUnary) {
lib/Target/PowerPC/PPCISelLowering.cpp:/// isVPKUWUMShuffleMask - Return true if this is the shuffle mask for a
lib/Target/PowerPC/PPCISelLowering.cpp:bool PPC::isVPKUWUMShuffleMask(ShuffleVectorSDNode *N, bool isUnary) {
lib/Target/PowerPC/PPCISelLowering.cpp:/// isVMerge - Common function, used to match vmrg* shuffles.
lib/Target/PowerPC/PPCISelLowering.cpp:static bool isVMerge(ShuffleVectorSDNode *N, unsigned UnitSize,
lib/Target/PowerPC/PPCISelLowering.cpp:         "PPC only supports shuffles by bytes!");
lib/Target/PowerPC/PPCISelLowering.cpp:/// isVMRGLShuffleMask - Return true if this is a shuffle mask suitable for
lib/Target/PowerPC/PPCISelLowering.cpp:bool PPC::isVMRGLShuffleMask(ShuffleVectorSDNode *N, unsigned UnitSize,
lib/Target/PowerPC/PPCISelLowering.cpp:/// isVMRGHShuffleMask - Return true if this is a shuffle mask suitable for
lib/Target/PowerPC/PPCISelLowering.cpp:bool PPC::isVMRGHShuffleMask(ShuffleVectorSDNode *N, unsigned UnitSize,
lib/Target/PowerPC/PPCISelLowering.cpp:/// isVSLDOIShuffleMask - If this is a vsldoi shuffle mask, return the shift
lib/Target/PowerPC/PPCISelLowering.cpp:int PPC::isVSLDOIShuffleMask(SDNode *N, bool isUnary) {
lib/Target/PowerPC/PPCISelLowering.cpp:         "PPC only supports shuffles by bytes!");
lib/Target/PowerPC/PPCISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(N);
lib/Target/PowerPC/PPCISelLowering.cpp:  // Find the first non-undef value in the shuffle mask.
lib/Target/PowerPC/PPCISelLowering.cpp:/// isSplatShuffleMask - Return true if the specified VECTOR_SHUFFLE operand
lib/Target/PowerPC/PPCISelLowering.cpp:bool PPC::isSplatShuffleMask(ShuffleVectorSDNode *N, unsigned EltSize) {
lib/Target/PowerPC/PPCISelLowering.cpp:/// specified isSplatShuffleMask VECTOR_SHUFFLE mask.
lib/Target/PowerPC/PPCISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(N);
lib/Target/PowerPC/PPCISelLowering.cpp:  assert(isSplatShuffleMask(SVOp, EltSize));
lib/Target/PowerPC/PPCISelLowering.cpp:  SDValue T = DAG.getVectorShuffle(MVT::v16i8, dl, LHS, RHS, Ops);
lib/Target/PowerPC/PPCISelLowering.cpp:/// GeneratePerfectShuffle - Given an entry in the perfect-shuffle table, emit
lib/Target/PowerPC/PPCISelLowering.cpp:/// the specified operations to build the shuffle.
lib/Target/PowerPC/PPCISelLowering.cpp:static SDValue GeneratePerfectShuffle(unsigned PFEntry, SDValue LHS,
lib/Target/PowerPC/PPCISelLowering.cpp:  OpLHS = GeneratePerfectShuffle(PerfectShuffleTable[LHSID], LHS, RHS, DAG, dl);
lib/Target/PowerPC/PPCISelLowering.cpp:  OpRHS = GeneratePerfectShuffle(PerfectShuffleTable[RHSID], LHS, RHS, DAG, dl);
lib/Target/PowerPC/PPCISelLowering.cpp:  SDValue T = DAG.getVectorShuffle(MVT::v16i8, dl, OpLHS, OpRHS, ShufIdxs);
lib/Target/PowerPC/PPCISelLowering.cpp:/// is a shuffle we can handle in a single instruction, return it.  Otherwise,
lib/Target/PowerPC/PPCISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
lib/Target/PowerPC/PPCISelLowering.cpp:    if (PPC::isSplatShuffleMask(SVOp, 1) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isSplatShuffleMask(SVOp, 2) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isSplatShuffleMask(SVOp, 4) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVPKUWUMShuffleMask(SVOp, true) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVPKUHUMShuffleMask(SVOp, true) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVSLDOIShuffleMask(SVOp, true) != -1 ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVMRGLShuffleMask(SVOp, 1, true) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVMRGLShuffleMask(SVOp, 2, true) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVMRGLShuffleMask(SVOp, 4, true) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVMRGHShuffleMask(SVOp, 1, true) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVMRGHShuffleMask(SVOp, 2, true) ||
lib/Target/PowerPC/PPCISelLowering.cpp:        PPC::isVMRGHShuffleMask(SVOp, 4, true)) {
lib/Target/PowerPC/PPCISelLowering.cpp:  // Altivec has a variety of "shuffle immediates" that take two vector inputs
lib/Target/PowerPC/PPCISelLowering.cpp:  if (PPC::isVPKUWUMShuffleMask(SVOp, false) ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVPKUHUMShuffleMask(SVOp, false) ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVSLDOIShuffleMask(SVOp, false) != -1 ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVMRGLShuffleMask(SVOp, 1, false) ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVMRGLShuffleMask(SVOp, 2, false) ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVMRGLShuffleMask(SVOp, 4, false) ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVMRGHShuffleMask(SVOp, 1, false) ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVMRGHShuffleMask(SVOp, 2, false) ||
lib/Target/PowerPC/PPCISelLowering.cpp:      PPC::isVMRGHShuffleMask(SVOp, 4, false))
lib/Target/PowerPC/PPCISelLowering.cpp:  // Check to see if this is a shuffle of 4-byte values.  If so, we can use our
lib/Target/PowerPC/PPCISelLowering.cpp:  // perfect shuffle table to emit an optimal matching sequence.
lib/Target/PowerPC/PPCISelLowering.cpp:  bool isFourElementShuffle = true;
lib/Target/PowerPC/PPCISelLowering.cpp:  for (unsigned i = 0; i != 4 && isFourElementShuffle; ++i) { // Element number
lib/Target/PowerPC/PPCISelLowering.cpp:        isFourElementShuffle = false;
lib/Target/PowerPC/PPCISelLowering.cpp:        isFourElementShuffle = false;
lib/Target/PowerPC/PPCISelLowering.cpp:  // If this shuffle can be expressed as a shuffle of 4-byte elements, use the
lib/Target/PowerPC/PPCISelLowering.cpp:  // perfect shuffle vector to determine if it is cost effective to do this as
lib/Target/PowerPC/PPCISelLowering.cpp:  if (isFourElementShuffle) {
lib/Target/PowerPC/PPCISelLowering.cpp:    // Compute the index in the perfect shuffle table.
lib/Target/PowerPC/PPCISelLowering.cpp:    unsigned PFEntry = PerfectShuffleTable[PFTableIndex];
lib/Target/PowerPC/PPCISelLowering.cpp:    // used (perhaps because there are multiple permutes with the same shuffle
lib/Target/PowerPC/PPCISelLowering.cpp:    // As a compromise, we only emit discrete instructions if the shuffle can be
lib/Target/PowerPC/PPCISelLowering.cpp:      return GeneratePerfectShuffle(PFEntry, V1, V2, DAG, dl);
lib/Target/PowerPC/PPCISelLowering.cpp:    return DAG.getVectorShuffle(MVT::v16i8, dl, EvenParts, OddParts, Ops);
lib/Target/PowerPC/PPCISelLowering.h:    /// isVPKUHUMShuffleMask - Return true if this is the shuffle mask for a
lib/Target/PowerPC/PPCISelLowering.h:    bool isVPKUHUMShuffleMask(ShuffleVectorSDNode *N, bool isUnary);
lib/Target/PowerPC/PPCISelLowering.h:    /// isVPKUWUMShuffleMask - Return true if this is the shuffle mask for a
lib/Target/PowerPC/PPCISelLowering.h:    bool isVPKUWUMShuffleMask(ShuffleVectorSDNode *N, bool isUnary);
lib/Target/PowerPC/PPCISelLowering.h:    /// isVMRGLShuffleMask - Return true if this is a shuffle mask suitable for
lib/Target/PowerPC/PPCISelLowering.h:    bool isVMRGLShuffleMask(ShuffleVectorSDNode *N, unsigned UnitSize,
lib/Target/PowerPC/PPCISelLowering.h:    /// isVMRGHShuffleMask - Return true if this is a shuffle mask suitable for
lib/Target/PowerPC/PPCISelLowering.h:    bool isVMRGHShuffleMask(ShuffleVectorSDNode *N, unsigned UnitSize,
lib/Target/PowerPC/PPCISelLowering.h:    /// isVSLDOIShuffleMask - If this is a vsldoi shuffle mask, return the shift
lib/Target/PowerPC/PPCISelLowering.h:    int isVSLDOIShuffleMask(SDNode *N, bool isUnary);
lib/Target/PowerPC/PPCISelLowering.h:    /// isSplatShuffleMask - Return true if the specified VECTOR_SHUFFLE operand
lib/Target/PowerPC/PPCISelLowering.h:    bool isSplatShuffleMask(ShuffleVectorSDNode *N, unsigned EltSize);
lib/Target/PowerPC/PPCISelLowering.h:    /// specified isSplatShuffleMask VECTOR_SHUFFLE mask.
lib/Target/PowerPC/PPCInstrAltivec.td:def vpkuhum_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                              (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVPKUHUMShuffleMask(cast<ShuffleVectorSDNode>(N), false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vpkuwum_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                              (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVPKUWUMShuffleMask(cast<ShuffleVectorSDNode>(N), false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vpkuhum_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                    (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVPKUHUMShuffleMask(cast<ShuffleVectorSDNode>(N), true);
lib/Target/PowerPC/PPCInstrAltivec.td:def vpkuwum_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                    (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVPKUWUMShuffleMask(cast<ShuffleVectorSDNode>(N), true);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrglb_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle (v16i8 node:$lhs), node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGLShuffleMask(cast<ShuffleVectorSDNode>(N), 1, false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrglh_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle (v16i8 node:$lhs), node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGLShuffleMask(cast<ShuffleVectorSDNode>(N), 2, false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrglw_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle (v16i8 node:$lhs), node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGLShuffleMask(cast<ShuffleVectorSDNode>(N), 4, false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrghb_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle (v16i8 node:$lhs), node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGHShuffleMask(cast<ShuffleVectorSDNode>(N), 1, false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrghh_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle (v16i8 node:$lhs), node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGHShuffleMask(cast<ShuffleVectorSDNode>(N), 2, false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrghw_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle (v16i8 node:$lhs), node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGHShuffleMask(cast<ShuffleVectorSDNode>(N), 4, false);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrglb_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                               (vector_shuffle (v16i8 node:$lhs), node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGLShuffleMask(cast<ShuffleVectorSDNode>(N), 1, true);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrglh_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                   (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGLShuffleMask(cast<ShuffleVectorSDNode>(N), 2, true);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrglw_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                   (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGLShuffleMask(cast<ShuffleVectorSDNode>(N), 4, true);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrghb_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                   (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGHShuffleMask(cast<ShuffleVectorSDNode>(N), 1, true);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrghh_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                   (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGHShuffleMask(cast<ShuffleVectorSDNode>(N), 2, true);
lib/Target/PowerPC/PPCInstrAltivec.td:def vmrghw_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                   (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVMRGHShuffleMask(cast<ShuffleVectorSDNode>(N), 4, true);
lib/Target/PowerPC/PPCInstrAltivec.td:def VSLDOI_get_imm : SDNodeXForm<vector_shuffle, [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return getI32Imm(PPC::isVSLDOIShuffleMask(N, false));
lib/Target/PowerPC/PPCInstrAltivec.td:def vsldoi_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVSLDOIShuffleMask(N, false) != -1;
lib/Target/PowerPC/PPCInstrAltivec.td:/// vector_shuffle(X,undef,mask) by the dag combiner.
lib/Target/PowerPC/PPCInstrAltivec.td:def VSLDOI_unary_get_imm : SDNodeXForm<vector_shuffle, [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return getI32Imm(PPC::isVSLDOIShuffleMask(N, true));
lib/Target/PowerPC/PPCInstrAltivec.td:def vsldoi_unary_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                                   (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isVSLDOIShuffleMask(N, true) != -1;
lib/Target/PowerPC/PPCInstrAltivec.td:// VSPLT*_get_imm xform function: convert vector_shuffle mask to VSPLT* imm.
lib/Target/PowerPC/PPCInstrAltivec.td:def VSPLTB_get_imm : SDNodeXForm<vector_shuffle, [{
lib/Target/PowerPC/PPCInstrAltivec.td:def vspltb_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isSplatShuffleMask(cast<ShuffleVectorSDNode>(N), 1);
lib/Target/PowerPC/PPCInstrAltivec.td:def VSPLTH_get_imm : SDNodeXForm<vector_shuffle, [{
lib/Target/PowerPC/PPCInstrAltivec.td:def vsplth_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isSplatShuffleMask(cast<ShuffleVectorSDNode>(N), 2);
lib/Target/PowerPC/PPCInstrAltivec.td:def VSPLTW_get_imm : SDNodeXForm<vector_shuffle, [{
lib/Target/PowerPC/PPCInstrAltivec.td:def vspltw_shuffle : PatFrag<(ops node:$lhs, node:$rhs),
lib/Target/PowerPC/PPCInstrAltivec.td:                             (vector_shuffle node:$lhs, node:$rhs), [{
lib/Target/PowerPC/PPCInstrAltivec.td:  return PPC::isSplatShuffleMask(cast<ShuffleVectorSDNode>(N), 4);
lib/Target/PowerPC/PPCInstrAltivec.td:// Shuffles.
lib/Target/PowerPC/PPCInstrAltivec.td:                         (vsldoi_shuffle:$SH v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                      [(set v16i8:$vD, (vmrghb_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                      [(set v16i8:$vD, (vmrghh_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                      [(set v16i8:$vD, (vmrghw_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                      [(set v16i8:$vD, (vmrglb_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                      [(set v16i8:$vD, (vmrglh_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                      [(set v16i8:$vD, (vmrglw_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                        (vspltb_shuffle:$UIMM v16i8:$vB, (undef)))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                        (vsplth_shuffle:$UIMM v16i8:$vB, (undef)))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                        (vspltw_shuffle:$UIMM v16i8:$vB, (undef)))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                         (vpkuhum_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:                         (vpkuwum_shuffle v16i8:$vA, v16i8:$vB))]>;
lib/Target/PowerPC/PPCInstrAltivec.td:// Shuffles.
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vsldoi_unary_shuffle:$in v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vpkuwum_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vpkuhum_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vmrglb_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vmrglh_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vmrglw_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vmrghb_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vmrghh_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCInstrAltivec.td:def:Pat<(vmrghw_unary_shuffle v16i8:$vA, undef),
lib/Target/PowerPC/PPCPerfectShuffle.h://===-- PPCPerfectShuffle.h - Altivec Perfect Shuffle Table -----*- C++ -*-===//
lib/Target/PowerPC/PPCPerfectShuffle.h:// This file, which was autogenerated by llvm-PerfectShuffle, contains data
lib/Target/PowerPC/PPCPerfectShuffle.h:// for the optimal way to build a perfect shuffle without using vperm.
lib/Target/PowerPC/PPCPerfectShuffle.h:static const unsigned PerfectShuffleTable[6561+1] = {
lib/Target/PowerPC/PPCTargetTransformInfo.cpp:  virtual unsigned getShuffleCost(ShuffleKind Kind, Type *Tp,
lib/Target/PowerPC/PPCTargetTransformInfo.cpp:unsigned PPCTTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
lib/Target/PowerPC/PPCTargetTransformInfo.cpp:  return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp);
lib/Target/PowerPC/README.txt:argument bytes for r4 and r5. The trick then would be to shuffle the argument
lib/Target/R600/R600TextureIntrinsicsReplacer.cpp:        Builder.CreateShuffleVector(Coord, Coord, SwizzleMask);
lib/Target/Sparc/SparcInstrVIS.td:def BSHUFFLE  : VISInst<0b000011100, "bshuffle">;
lib/Target/X86/README-FPStack.txt:The FP stackifier should handle simple permutates to reduce number of shuffle
lib/Target/X86/README-SSE.txt:small table + unaligned load + shuffle instead of going through memory.
lib/Target/X86/README-SSE.txt:  return _mm_shuffle_epi8(value,
lib/Target/X86/README-SSE.txt:Better codegen for vector_shuffles like this { x, 0, 0, 0 } or { x, 0, x, 0}.
lib/Target/X86/README-SSE.txt:This can be used to simplify a variety of shuffle operations, where the
lib/Target/X86/README-SSE.txt:We should transform a shuffle of two vectors of constants into a single vector
lib/Target/X86/README-SSE.txt:doing a shuffle from v[1] to v[0] then a float store.
lib/Target/X86/X86FloatingPoint.cpp:    // Shuffle live registers to match the expectations of successor blocks.
lib/Target/X86/X86FloatingPoint.cpp:    /// Shuffle the top FixCount stack entries such that FP reg FixStack[0] is
lib/Target/X86/X86FloatingPoint.cpp:    void shuffleStackTop(const unsigned char *FixStack, unsigned FixCount,
lib/Target/X86/X86FloatingPoint.cpp:/// MBB. Shuffle live registers to match the expected fixed stack of any
lib/Target/X86/X86FloatingPoint.cpp:  // FIXME: This can probably be combined with the shuffle below.
lib/Target/X86/X86FloatingPoint.cpp:    shuffleStackTop(Bundle.FixStack, Bundle.FixCount, Term);
lib/Target/X86/X86FloatingPoint.cpp:/// shuffleStackTop - emit fxch instructions before I to shuffle the top
lib/Target/X86/X86FloatingPoint.cpp:void FPS::shuffleStackTop(const unsigned char *FixStack,
lib/Target/X86/X86FloatingPoint.cpp:    shuffleStackTop(PendingST, NumPendingSTs, I);
lib/Target/X86/X86ISelDAGToDAG.cpp:  // elements.  This is a vector shuffle from the zero vector.
lib/Target/X86/X86ISelLowering.cpp:#include "Utils/X86ShuffleDecode.h"
lib/Target/X86/X86ISelLowering.cpp:    // Custom lower build_vector, vector_shuffle, and extract_vector_elt.
lib/Target/X86/X86ISelLowering.cpp:static bool isTargetShuffle(unsigned Opcode) {
lib/Target/X86/X86ISelLowering.cpp:static SDValue getTargetShuffleNode(unsigned Opc, SDLoc dl, EVT VT,
lib/Target/X86/X86ISelLowering.cpp:  default: llvm_unreachable("Unknown x86 shuffle node");
lib/Target/X86/X86ISelLowering.cpp:static SDValue getTargetShuffleNode(unsigned Opc, SDLoc dl, EVT VT,
lib/Target/X86/X86ISelLowering.cpp:  default: llvm_unreachable("Unknown x86 shuffle node");
lib/Target/X86/X86ISelLowering.cpp:static SDValue getTargetShuffleNode(unsigned Opc, SDLoc dl, EVT VT,
lib/Target/X86/X86ISelLowering.cpp:  default: llvm_unreachable("Unknown x86 shuffle node");
lib/Target/X86/X86ISelLowering.cpp:static SDValue getTargetShuffleNode(unsigned Opc, SDLoc dl, EVT VT,
lib/Target/X86/X86ISelLowering.cpp:  default: llvm_unreachable("Unknown x86 shuffle node");
lib/Target/X86/X86ISelLowering.cpp:/// isPSHUFDMask - Return true if the node specifies a shuffle of elements that
lib/Target/X86/X86ISelLowering.cpp:/// isPSHUFHWMask - Return true if the node specifies a shuffle of elements that
lib/Target/X86/X86ISelLowering.cpp:  // Upper quadword shuffled.
lib/Target/X86/X86ISelLowering.cpp:    // Upper quadword shuffled.
lib/Target/X86/X86ISelLowering.cpp:/// isPSHUFLWMask - Return true if the node specifies a shuffle of elements that
lib/Target/X86/X86ISelLowering.cpp:  // Lower quadword shuffled.
lib/Target/X86/X86ISelLowering.cpp:    // Lower quadword shuffled.
lib/Target/X86/X86ISelLowering.cpp:/// isPALIGNRMask - Return true if the node specifies a shuffle of elements that
lib/Target/X86/X86ISelLowering.cpp:  // Do not handle 64-bit element shuffles with palignr.
lib/Target/X86/X86ISelLowering.cpp:/// CommuteVectorShuffleMask - Change values in a shuffle permute mask assuming
lib/Target/X86/X86ISelLowering.cpp:static void CommuteVectorShuffleMask(SmallVectorImpl<int> &Mask,
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to 128/256-bit
lib/Target/X86/X86ISelLowering.cpp:/// reverse of what x86 shuffles want.
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to MOVHLPS.
lib/Target/X86/X86ISelLowering.cpp:/// of vector_shuffle v, v, <2, 3, 2, 3>, i.e. vector_shuffle v, undef,
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to MOVLP{S|D}.
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to MOVLHPS.
lib/Target/X86/X86ISelLowering.cpp:SDValue Compact8x32ShuffleNode(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:    Op1 = DAG.getVectorShuffle(VT, dl, Op1, UndefNode, ShiftRightMask);
lib/Target/X86/X86ISelLowering.cpp:    Op0 = DAG.getVectorShuffle(VT, dl, Op0, UndefNode, ShiftLeftMask);
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(VT, dl, Op0, Op1, BlendMask);
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to UNPCKL.
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to UNPCKH.
lib/Target/X86/X86ISelLowering.cpp:/// of vector_shuffle v, v, <0, 4, 1, 5>, i.e. vector_shuffle v, undef,
lib/Target/X86/X86ISelLowering.cpp:/// of vector_shuffle v, v, <2, 6, 3, 7>, i.e. vector_shuffle v, undef,
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to MOVSS,
lib/Target/X86/X86ISelLowering.cpp:/// isVPERM2X128Mask - Match 256-bit shuffles where the elements are considered
lib/Target/X86/X86ISelLowering.cpp:/// shuffle bellow:
lib/Target/X86/X86ISelLowering.cpp:///   vector_shuffle <4, 5, 6, 7, 12, 13, 14, 15>
lib/Target/X86/X86ISelLowering.cpp:  // The shuffle result is divided into half A and half B. In total the two
lib/Target/X86/X86ISelLowering.cpp:/// getShuffleVPERM2X128Immediate - Return the appropriate immediate to shuffle
lib/Target/X86/X86ISelLowering.cpp:static unsigned getShuffleVPERM2X128Immediate(ShuffleVectorSDNode *SVOp) {
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to VPERMILPD*.
lib/Target/X86/X86ISelLowering.cpp:/// In VPERMILPD the two lanes could be shuffled independently of each other
lib/Target/X86/X86ISelLowering.cpp:/// isCommutedMOVLMask - Returns true if the shuffle mask is except the reverse
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to MOVSHDUP.
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to MOVSLDUP.
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to 256-bit
lib/Target/X86/X86ISelLowering.cpp:/// specifies a shuffle of elements that is suitable for input to 128-bit
lib/Target/X86/X86ISelLowering.cpp:/// getShuffleSHUFImmediate - Return the appropriate immediate to shuffle
lib/Target/X86/X86ISelLowering.cpp:static unsigned getShuffleSHUFImmediate(ShuffleVectorSDNode *N) {
lib/Target/X86/X86ISelLowering.cpp:/// getShufflePSHUFHWImmediate - Return the appropriate immediate to shuffle
lib/Target/X86/X86ISelLowering.cpp:static unsigned getShufflePSHUFHWImmediate(ShuffleVectorSDNode *N) {
lib/Target/X86/X86ISelLowering.cpp:/// getShufflePSHUFLWImmediate - Return the appropriate immediate to shuffle
lib/Target/X86/X86ISelLowering.cpp:static unsigned getShufflePSHUFLWImmediate(ShuffleVectorSDNode *N) {
lib/Target/X86/X86ISelLowering.cpp:/// getShufflePALIGNRImmediate - Return the appropriate immediate to shuffle
lib/Target/X86/X86ISelLowering.cpp:static unsigned getShufflePALIGNRImmediate(ShuffleVectorSDNode *SVOp) {
lib/Target/X86/X86ISelLowering.cpp:/// CommuteVectorShuffle - Swap vector_shuffle operands as well as values in
lib/Target/X86/X86ISelLowering.cpp:static SDValue CommuteVectorShuffle(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(VT, SDLoc(SVOp), SVOp->getOperand(1),
lib/Target/X86/X86ISelLowering.cpp:/// isZeroShuffle - Returns true if N is a VECTOR_SHUFFLE that can be resolved
lib/Target/X86/X86ISelLowering.cpp:/// FIXME: move to dag combiner / method on ShuffleVectorSDNode
lib/Target/X86/X86ISelLowering.cpp:static bool isZeroShuffle(ShuffleVectorSDNode *N) {
lib/Target/X86/X86ISelLowering.cpp:/// getMOVLMask - Returns a vector_shuffle mask for an movs{s|d}, movd
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(VT, dl, V1, V2, &Mask[0]);
lib/Target/X86/X86ISelLowering.cpp:/// getUnpackl - Returns a vector_shuffle node for an unpackl operation.
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(VT, dl, V1, V2, &Mask[0]);
lib/Target/X86/X86ISelLowering.cpp:/// getUnpackh - Returns a vector_shuffle node for an unpackh operation.
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(VT, dl, V1, V2, &Mask[0]);
lib/Target/X86/X86ISelLowering.cpp:// a generic shuffle instruction because the target has no such instructions.
lib/Target/X86/X86ISelLowering.cpp:// Generate shuffles which repeat i16 and i8 several times until they can be
lib/Target/X86/X86ISelLowering.cpp:// represented by v4f32 and then be manipulated by target suported shuffles.
lib/Target/X86/X86ISelLowering.cpp:/// getLegalSplat - Generate a legal splat with supported x86 shuffles
lib/Target/X86/X86ISelLowering.cpp:    V = DAG.getVectorShuffle(MVT::v4f32, dl, V, DAG.getUNDEF(MVT::v4f32),
lib/Target/X86/X86ISelLowering.cpp:    V = DAG.getVectorShuffle(MVT::v8f32, dl, V, DAG.getUNDEF(MVT::v8f32),
lib/Target/X86/X86ISelLowering.cpp:/// PromoteSplat - Splat is promoted to target supported vector shuffles.
lib/Target/X86/X86ISelLowering.cpp:static SDValue PromoteSplat(ShuffleVectorSDNode *SV, SelectionDAG &DAG) {
lib/Target/X86/X86ISelLowering.cpp:  // All i16 and i8 vector types can't be used directly by a generic shuffle
lib/Target/X86/X86ISelLowering.cpp:  // instruction because the target has no such instruction. Generate shuffles
lib/Target/X86/X86ISelLowering.cpp:  // be manipulated by target suported shuffles.
lib/Target/X86/X86ISelLowering.cpp:  // to use VPERM* to shuffle the vectors
lib/Target/X86/X86ISelLowering.cpp:/// getShuffleVectorZeroOrUndef - Return a vector_shuffle of the specified
lib/Target/X86/X86ISelLowering.cpp:/// vector of zero or undef vector.  This produces a shuffle where the low
lib/Target/X86/X86ISelLowering.cpp:/// Idx.  This produces a shuffle mask like 4,1,2,3 (idx=0) or  0,1,2,4 (idx=3).
lib/Target/X86/X86ISelLowering.cpp:static SDValue getShuffleVectorZeroOrUndef(SDValue V2, unsigned Idx,
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(VT, SDLoc(V2), V1, V2, &MaskVec[0]);
lib/Target/X86/X86ISelLowering.cpp:/// getTargetShuffleMask - Calculates the shuffle mask corresponding to the
lib/Target/X86/X86ISelLowering.cpp:static bool getTargetShuffleMask(SDNode *N, MVT VT,
lib/Target/X86/X86ISelLowering.cpp:  default: llvm_unreachable("unknown target shuffle node");
lib/Target/X86/X86ISelLowering.cpp:/// getShuffleScalarElt - Returns the scalar element that will make up the ith
lib/Target/X86/X86ISelLowering.cpp:/// element of the result of the vector shuffle.
lib/Target/X86/X86ISelLowering.cpp:static SDValue getShuffleScalarElt(SDNode *N, unsigned Index, SelectionDAG &DAG,
lib/Target/X86/X86ISelLowering.cpp:  if (const ShuffleVectorSDNode *SV = dyn_cast<ShuffleVectorSDNode>(N)) {
lib/Target/X86/X86ISelLowering.cpp:    return getShuffleScalarElt(NewV.getNode(), Elt % NumElems, DAG, Depth+1);
lib/Target/X86/X86ISelLowering.cpp:  // Recurse into target specific vector shuffles to find scalars.
lib/Target/X86/X86ISelLowering.cpp:  if (isTargetShuffle(Opcode)) {
lib/Target/X86/X86ISelLowering.cpp:    SmallVector<int, 16> ShuffleMask;
lib/Target/X86/X86ISelLowering.cpp:    if (!getTargetShuffleMask(N, ShufVT, ShuffleMask, IsUnary))
lib/Target/X86/X86ISelLowering.cpp:    int Elt = ShuffleMask[Index];
lib/Target/X86/X86ISelLowering.cpp:    return getShuffleScalarElt(NewV.getNode(), Elt % NumElems, DAG,
lib/Target/X86/X86ISelLowering.cpp:/// shuffle operation which come from a consecutively from a zero. The
lib/Target/X86/X86ISelLowering.cpp:static unsigned getNumOfConsecutiveZeros(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:    SDValue Elt = getShuffleScalarElt(SVOp, Index, DAG, 0);
lib/Target/X86/X86ISelLowering.cpp:/// isShuffleMaskConsecutive - Check if the shuffle mask indicies [MaskI, MaskE)
lib/Target/X86/X86ISelLowering.cpp:bool isShuffleMaskConsecutive(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:/// isVectorShiftRight - Returns true if the shuffle can be implemented as a
lib/Target/X86/X86ISelLowering.cpp:static bool isVectorShiftRight(ShuffleVectorSDNode *SVOp, SelectionDAG &DAG,
lib/Target/X86/X86ISelLowering.cpp:  //   vector_shuffle V1, V2 <1, 2, 3, X>
lib/Target/X86/X86ISelLowering.cpp:  if (!isShuffleMaskConsecutive(SVOp,
lib/Target/X86/X86ISelLowering.cpp:/// isVectorShiftLeft - Returns true if the shuffle can be implemented as a
lib/Target/X86/X86ISelLowering.cpp:static bool isVectorShiftLeft(ShuffleVectorSDNode *SVOp, SelectionDAG &DAG,
lib/Target/X86/X86ISelLowering.cpp:  //   vector_shuffle V1, V2 <X, X, 4, 5>
lib/Target/X86/X86ISelLowering.cpp:  if (!isShuffleMaskConsecutive(SVOp,
lib/Target/X86/X86ISelLowering.cpp:/// isVectorShift - Returns true if the shuffle can be implemented as a
lib/Target/X86/X86ISelLowering.cpp:static bool isVectorShift(ShuffleVectorSDNode *SVOp, SelectionDAG &DAG,
lib/Target/X86/X86ISelLowering.cpp:  // the shuffle mask.
lib/Target/X86/X86ISelLowering.cpp:    return DAG.getVectorShuffle(NVT, dl, V1, DAG.getUNDEF(NVT), &Mask[0]);
lib/Target/X86/X86ISelLowering.cpp:/// 2. A splat shuffle which uses a scalar_to_vector node which comes from
lib/Target/X86/X86ISelLowering.cpp:      ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
lib/Target/X86/X86ISelLowering.cpp:      // Shuffles must have a splat mask where the first element is
lib/Target/X86/X86ISelLowering.cpp:static SDValue buildFromShuffleMostly(SDValue Op, SelectionDAG &DAG) {
lib/Target/X86/X86ISelLowering.cpp:        // Quit if more than 2 vectors to shuffle
lib/Target/X86/X86ISelLowering.cpp:  SDValue NV = DAG.getVectorShuffle(VT, DL, VecIn1, VecIn2, &Mask[0]);
lib/Target/X86/X86ISelLowering.cpp:    // a constant pool load than it is to do a movd + shuffle.
lib/Target/X86/X86ISelLowering.cpp:        // convert it to a vector with movd (S2V+shuffle to zero extend).
lib/Target/X86/X86ISelLowering.cpp:        Item = getShuffleVectorZeroOrUndef(Item, 0, true, Subtarget, DAG);
lib/Target/X86/X86ISelLowering.cpp:          Item = DAG.getVectorShuffle(VecVT, dl, Item, DAG.getUNDEF(VecVT),
lib/Target/X86/X86ISelLowering.cpp:    // a vector, we can do this with SCALAR_TO_VECTOR + shuffle of zero into
lib/Target/X86/X86ISelLowering.cpp:        return getShuffleVectorZeroOrUndef(Item, 0, true, Subtarget, DAG);
lib/Target/X86/X86ISelLowering.cpp:          Item = getShuffleVectorZeroOrUndef(Item, 0, true, Subtarget, DAG);
lib/Target/X86/X86ISelLowering.cpp:    // movd/movss) to move this into the low element, then shuffle it into
lib/Target/X86/X86ISelLowering.cpp:      // Turn it into a shuffle of zero and zero-extended scalar to vector.
lib/Target/X86/X86ISelLowering.cpp:      Item = getShuffleVectorZeroOrUndef(Item, 0, NumZero > 0, Subtarget, DAG);
lib/Target/X86/X86ISelLowering.cpp:      return DAG.getVectorShuffle(VT, dl, Item, DAG.getUNDEF(VT), &MaskVec[0]);
lib/Target/X86/X86ISelLowering.cpp:  // Splat is obviously ok. Let legalizer expand it to a shuffle.
lib/Target/X86/X86ISelLowering.cpp:      // Instead of a shuffle like this:
lib/Target/X86/X86ISelLowering.cpp:      // shuffle (scalar_to_vector (load (ptr + 4))), undef, <0, 0, 0, 0>
lib/Target/X86/X86ISelLowering.cpp:      // shuffle (vload ptr)), undef, <1, 1, 1, 1>
lib/Target/X86/X86ISelLowering.cpp:  // shuffles to put them in place.
lib/Target/X86/X86ISelLowering.cpp:      return getShuffleVectorZeroOrUndef(V2, Idx, true, Subtarget, DAG);
lib/Target/X86/X86ISelLowering.cpp:  // If element VT is == 32 bits, turn it into a number of shuffles.
lib/Target/X86/X86ISelLowering.cpp:    return DAG.getVectorShuffle(VT, dl, V[0], V[1], &MaskVec[0]);
lib/Target/X86/X86ISelLowering.cpp:    // Check for a build vector from mostly shuffle plus few inserting.
lib/Target/X86/X86ISelLowering.cpp:    SDValue Sh = buildFromShuffleMostly(Op, DAG);
lib/Target/X86/X86ISelLowering.cpp:        // then it is safe to just drop this shuffle: V[i] is already in the
lib/Target/X86/X86ISelLowering.cpp:// Try to lower a shuffle node into a simple blend instruction.
lib/Target/X86/X86ISelLowering.cpp:LowerVECTOR_SHUFFLEtoBlend(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:// v8i16 shuffles - Prefer shuffles in the following order:
lib/Target/X86/X86ISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
lib/Target/X86/X86ISelLowering.cpp:  // of the two input vectors, shuffle them into one input vector so only a
lib/Target/X86/X86ISelLowering.cpp:  // If BestLoQuad or BestHiQuad are set, shuffle the quads together and update
lib/Target/X86/X86ISelLowering.cpp:  // the shuffle mask.  If a quad is scored as -1, that means that it contains
lib/Target/X86/X86ISelLowering.cpp:    NewV = DAG.getVectorShuffle(MVT::v2i64, dl,
lib/Target/X86/X86ISelLowering.cpp:    // source words for the shuffle, to aid later transformations.
lib/Target/X86/X86ISelLowering.cpp:    // pshufhw, that's as cheap as it gets.  Return the new shuffle.
lib/Target/X86/X86ISelLowering.cpp:      NewV = DAG.getVectorShuffle(MVT::v8i16, dl, NewV,
lib/Target/X86/X86ISelLowering.cpp:      ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(NewV.getNode());
lib/Target/X86/X86ISelLowering.cpp:      TargetMask = pshufhw ? getShufflePSHUFHWImmediate(SVOp):
lib/Target/X86/X86ISelLowering.cpp:                             getShufflePSHUFLWImmediate(SVOp);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(Opc, dl, MVT::v8i16, V1, TargetMask, DAG);
lib/Target/X86/X86ISelLowering.cpp:    // shuffle mask element to zero out elements that come from V2 in the V1
lib/Target/X86/X86ISelLowering.cpp:    // Calculate the shuffle mask for the second input, shuffle it, and
lib/Target/X86/X86ISelLowering.cpp:    // OR it with the first shuffled input.
lib/Target/X86/X86ISelLowering.cpp:    NewV = DAG.getVectorShuffle(MVT::v8i16, dl, NewV, DAG.getUNDEF(MVT::v8i16),
lib/Target/X86/X86ISelLowering.cpp:      ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(NewV.getNode());
lib/Target/X86/X86ISelLowering.cpp:      NewV = getTargetShuffleNode(X86ISD::PSHUFLW, dl, MVT::v8i16,
lib/Target/X86/X86ISelLowering.cpp:                                  getShufflePSHUFLWImmediate(SVOp), DAG);
lib/Target/X86/X86ISelLowering.cpp:    NewV = DAG.getVectorShuffle(MVT::v8i16, dl, NewV, DAG.getUNDEF(MVT::v8i16),
lib/Target/X86/X86ISelLowering.cpp:      ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(NewV.getNode());
lib/Target/X86/X86ISelLowering.cpp:      NewV = getTargetShuffleNode(X86ISD::PSHUFHW, dl, MVT::v8i16,
lib/Target/X86/X86ISelLowering.cpp:                                  getShufflePSHUFHWImmediate(SVOp), DAG);
lib/Target/X86/X86ISelLowering.cpp:// v16i8 shuffles - Prefer shuffles in the following order:
lib/Target/X86/X86ISelLowering.cpp:// 3. [all]   v8i16 shuffle + N x pextrw + rotate + pinsrw
lib/Target/X86/X86ISelLowering.cpp:static SDValue LowerVECTOR_SHUFFLEv16i8(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:    // Calculate the shuffle mask for the second input, shuffle it, and
lib/Target/X86/X86ISelLowering.cpp:    // OR it with the first shuffled input.
lib/Target/X86/X86ISelLowering.cpp:// v32i8 shuffles - Translate to VPSHUFB if possible.
lib/Target/X86/X86ISelLowering.cpp:SDValue LowerVECTOR_SHUFFLEv32i8(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:    CommuteVectorShuffleMask(MaskVals, 32);
lib/Target/X86/X86ISelLowering.cpp:/// RewriteAsNarrowerShuffle - Try rewriting v8i16 and v16i8 shuffles as 4 wide
lib/Target/X86/X86ISelLowering.cpp:/// done when every pair / quad of shuffle mask elements point to elements in
lib/Target/X86/X86ISelLowering.cpp:/// vector_shuffle X, Y, <2, 3, | 10, 11, | 0, 1, | 14, 15>
lib/Target/X86/X86ISelLowering.cpp:SDValue RewriteAsNarrowerShuffle(ShuffleVectorSDNode *SVOp,
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(NewVT, dl, V1, V2, &MaskVec[0]);
lib/Target/X86/X86ISelLowering.cpp:/// LowerVECTOR_SHUFFLE_256 - Handle all 256-bit wide vectors shuffles
lib/Target/X86/X86ISelLowering.cpp:/// which could not be matched by any known target speficic shuffle
lib/Target/X86/X86ISelLowering.cpp:LowerVECTOR_SHUFFLE_256(ShuffleVectorSDNode *SVOp, SelectionDAG &DAG) {
lib/Target/X86/X86ISelLowering.cpp:  SDValue NewOp = Compact8x32ShuffleNode(SVOp, DAG);
lib/Target/X86/X86ISelLowering.cpp:    // Build a shuffle mask for the output, discovering on the fly which
lib/Target/X86/X86ISelLowering.cpp:    // input vectors to use as shuffle operands (recorded in InputUsed).
lib/Target/X86/X86ISelLowering.cpp:    // If building a suitable shuffle vector proves too hard, then bail
lib/Target/X86/X86ISelLowering.cpp:      // Find or create a shuffle vector operand to hold this input.
lib/Target/X86/X86ISelLowering.cpp:        // shuffle vector.  Insert all elements into a BUILD_VECTOR instead.
lib/Target/X86/X86ISelLowering.cpp:      // Add the mask index for the new shuffle vector.
lib/Target/X86/X86ISelLowering.cpp:      // At least one input vector was used. Create a new shuffle vector.
lib/Target/X86/X86ISelLowering.cpp:      Output[l] = DAG.getVectorShuffle(NVT, dl, Op0, Op1, &Mask[0]);
lib/Target/X86/X86ISelLowering.cpp:/// 4 elements, and match them with several different shuffle types.
lib/Target/X86/X86ISelLowering.cpp:LowerVECTOR_SHUFFLE_128v4(ShuffleVectorSDNode *SVOp, SelectionDAG &DAG) {
lib/Target/X86/X86ISelLowering.cpp:    // implemented with two shuffles. First shuffle gather the elements.
lib/Target/X86/X86ISelLowering.cpp:    // The second shuffle, which takes the first shuffle as both of its
lib/Target/X86/X86ISelLowering.cpp:    V1 = DAG.getVectorShuffle(VT, dl, V1, V2, &Mask1[0]);
lib/Target/X86/X86ISelLowering.cpp:    return DAG.getVectorShuffle(VT, dl, V1, V1, &Mask2[0]);
lib/Target/X86/X86ISelLowering.cpp:      CommuteVectorShuffleMask(PermMask, 4);
lib/Target/X86/X86ISelLowering.cpp:    V2 = DAG.getVectorShuffle(VT, dl, V1, V2, &Mask1[0]);
lib/Target/X86/X86ISelLowering.cpp:      return DAG.getVectorShuffle(VT, dl, V1, V2, &Mask1[0]);
lib/Target/X86/X86ISelLowering.cpp:    return DAG.getVectorShuffle(VT, dl, V2, V1, &Mask1[0]);
lib/Target/X86/X86ISelLowering.cpp:  // Break it into (shuffle shuffle_hi, shuffle_lo).
lib/Target/X86/X86ISelLowering.cpp:  SDValue LoShuffle = DAG.getVectorShuffle(VT, dl, V1, V2, &LoMask[0]);
lib/Target/X86/X86ISelLowering.cpp:  SDValue HiShuffle = DAG.getVectorShuffle(VT, dl, V1, V2, &HiMask[0]);
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getVectorShuffle(VT, dl, LoShuffle, HiShuffle, &MaskOps[0]);
lib/Target/X86/X86ISelLowering.cpp:                     getTargetShuffleNode(X86ISD::MOVDDUP, dl, MVT::v2f64,
lib/Target/X86/X86ISelLowering.cpp:  assert(VT != MVT::v2i64 && "unsupported shuffle type");
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::MOVLHPD, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:                     getTargetShuffleNode(X86ISD::MOVLHPS, dl, MVT::v4f32,
lib/Target/X86/X86ISelLowering.cpp:         "unsupported shuffle type");
lib/Target/X86/X86ISelLowering.cpp:  return getTargetShuffleNode(X86ISD::MOVHLPS, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::MOVLPD, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:        return getTargetShuffleNode(X86ISD::MOVLPS, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:  // this is horrible, but will stay like this until we move all shuffle
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::MOVSD, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::MOVSS, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:  assert(VT != MVT::v4i32 && "unsupported shuffle type");
lib/Target/X86/X86ISelLowering.cpp:  return getTargetShuffleNode(X86ISD::SHUFP, dl, VT, V2, V1,
lib/Target/X86/X86ISelLowering.cpp:                              getShuffleSHUFImmediate(SVOp), DAG);
lib/Target/X86/X86ISelLowering.cpp:// Reduce a vector shuffle to zext.
lib/Target/X86/X86ISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
lib/Target/X86/X86ISelLowering.cpp:  // Check the shuffle mask.
lib/Target/X86/X86ISelLowering.cpp:  // Simplify the operand as it's prepared to be fed into shuffle.
lib/Target/X86/X86ISelLowering.cpp:NormalizeVectorShuffle(SDValue Op, const X86Subtarget *Subtarget,
lib/Target/X86/X86ISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
lib/Target/X86/X86ISelLowering.cpp:  if (isZeroShuffle(SVOp))
lib/Target/X86/X86ISelLowering.cpp:  // Check integer expanding shuffles.
lib/Target/X86/X86ISelLowering.cpp:  // If the shuffle can be profitably rewritten as a narrower shuffle, then
lib/Target/X86/X86ISelLowering.cpp:    SDValue NewOp = RewriteAsNarrowerShuffle(SVOp, DAG);
lib/Target/X86/X86ISelLowering.cpp:      SDValue NewOp = RewriteAsNarrowerShuffle(SVOp, DAG);
lib/Target/X86/X86ISelLowering.cpp:        if (isCommutedMOVLMask(cast<ShuffleVectorSDNode>(NewOp)->getMask(),
lib/Target/X86/X86ISelLowering.cpp:      SDValue NewOp = RewriteAsNarrowerShuffle(SVOp, DAG);
lib/Target/X86/X86ISelLowering.cpp:        if (isMOVLMask(cast<ShuffleVectorSDNode>(NewOp)->getMask(), NewVT))
lib/Target/X86/X86ISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(Op);
lib/Target/X86/X86ISelLowering.cpp:  assert(VT.getSizeInBits() != 64 && "Can't lower MMX shuffles");
lib/Target/X86/X86ISelLowering.cpp:  // When we create a shuffle node we put the UNDEF node to second operand,
lib/Target/X86/X86ISelLowering.cpp:    return CommuteVectorShuffle(SVOp, DAG);
lib/Target/X86/X86ISelLowering.cpp:  // Vector shuffle lowering takes 3 steps:
lib/Target/X86/X86ISelLowering.cpp:  // 2) Matching of shuffles with known shuffle masks to x86 target specific
lib/Target/X86/X86ISelLowering.cpp:  //    shuffle nodes.
lib/Target/X86/X86ISelLowering.cpp:  // 3) Rewriting of unmatched masks into new generic shuffle operations,
lib/Target/X86/X86ISelLowering.cpp:  //    so the shuffle can be broken into other shuffles and the legalizer can
lib/Target/X86/X86ISelLowering.cpp:  // The general idea is that no vector_shuffle operation should be left to
lib/Target/X86/X86ISelLowering.cpp:  SDValue NewOp = NormalizeVectorShuffle(Op, Subtarget, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::UNPCKL, dl, VT, V1, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::UNPCKH, dl, VT, V1, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::UNPCKH, dl, VT, V1, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::MOVLHPS, dl, VT, V1, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:    unsigned TargetMask = getShuffleSHUFImmediate(SVOp);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::PSHUFD, dl, VT, V1, TargetMask, DAG);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::VPERMILP, dl, VT, V1, TargetMask,
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::SHUFP, dl, VT, V1, V1,
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::PALIGNR, dl, VT, V1, V2,
lib/Target/X86/X86ISelLowering.cpp:                                getShufflePALIGNRImmediate(SVOp),
lib/Target/X86/X86ISelLowering.cpp:        return getTargetShuffleNode(X86ISD::MOVSD, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:        return getTargetShuffleNode(X86ISD::MOVSS, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::MOVSHDUP, dl, VT, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::MOVSLDUP, dl, VT, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return CommuteVectorShuffle(SVOp, DAG);
lib/Target/X86/X86ISelLowering.cpp:    CommuteVectorShuffleMask(M, NumElems);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::UNPCKL, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::UNPCKH, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:    // new vector_shuffle with the corrected mask.p
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::UNPCKL, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::UNPCKH, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:    CommuteVectorShuffleMask(M, NumElems);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::UNPCKL, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::UNPCKH, dl, VT, V1, V2, DAG);
lib/Target/X86/X86ISelLowering.cpp:  // Normalize the node to match x86 shuffle ops if needed
lib/Target/X86/X86ISelLowering.cpp:    return CommuteVectorShuffle(SVOp, DAG);
lib/Target/X86/X86ISelLowering.cpp:  // The checks below are all present in isShuffleMaskLegal, but they are
lib/Target/X86/X86ISelLowering.cpp:  if (ShuffleVectorSDNode::isSplatMask(&M[0], VT) &&
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::UNPCKL, dl, VT, V1, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::PSHUFHW, dl, VT, V1,
lib/Target/X86/X86ISelLowering.cpp:                                getShufflePSHUFHWImmediate(SVOp),
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::PSHUFLW, dl, VT, V1,
lib/Target/X86/X86ISelLowering.cpp:                                getShufflePSHUFLWImmediate(SVOp),
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::SHUFP, dl, VT, V1, V2,
lib/Target/X86/X86ISelLowering.cpp:                                getShuffleSHUFImmediate(SVOp), DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::UNPCKL, dl, VT, V1, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::UNPCKH, dl, VT, V1, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:  // Generate target specific nodes for 128 or 256-bit shuffles only
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::MOVDDUP, dl, VT, V1, DAG);
lib/Target/X86/X86ISelLowering.cpp:      return getTargetShuffleNode(X86ISD::PSHUFD, dl, VT, V1,
lib/Target/X86/X86ISelLowering.cpp:                                  getShuffleSHUFImmediate(SVOp), DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::VPERMILP, dl, VT, V1,
lib/Target/X86/X86ISelLowering.cpp:                                getShuffleSHUFImmediate(SVOp), DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::VPERM2X128, dl, VT, V1,
lib/Target/X86/X86ISelLowering.cpp:                                V2, getShuffleVPERM2X128Immediate(SVOp), DAG);
lib/Target/X86/X86ISelLowering.cpp:    return getTargetShuffleNode(X86ISD::VPERMI, dl, VT, V1, Imm8, DAG);
lib/Target/X86/X86ISelLowering.cpp:  // Since no target specific shuffle was selected for this generic one,
lib/Target/X86/X86ISelLowering.cpp:  // lower it into other known shuffles. FIXME: this isn't true yet, but
lib/Target/X86/X86ISelLowering.cpp:  // several different shuffle types.
lib/Target/X86/X86ISelLowering.cpp:  // Handle general 256-bit shuffles
lib/Target/X86/X86ISelLowering.cpp:    SDValue Vec = DAG.getVectorShuffle(VVT, dl, Op.getOperand(0),
lib/Target/X86/X86ISelLowering.cpp:    SDValue Vec = DAG.getVectorShuffle(VVT, dl, Op.getOperand(0),
lib/Target/X86/X86ISelLowering.cpp:    SDValue Shuffle = getTargetShuffleNode(X86ISD::PSHUFD, dl, MVT::v4i32,
lib/Target/X86/X86ISelLowering.cpp:                         DAG.getNode(ISD::BITCAST, dl, MVT::v2f64, Shuffle),
lib/Target/X86/X86ISelLowering.cpp:  Load = getShuffleVectorZeroOrUndef(Load, 0, true, Subtarget, DAG);
lib/Target/X86/X86ISelLowering.cpp:      In = DAG.getVectorShuffle(MVT::v8i32, DL, In, DAG.getUNDEF(MVT::v8i32),
lib/Target/X86/X86ISelLowering.cpp:    return DAG.getVectorShuffle(VT, DL, OpLo, OpHi, ShufMask);
lib/Target/X86/X86ISelLowering.cpp:      In = DAG.getVectorShuffle(MVT::v4i64, DL,  In, DAG.getUNDEF(MVT::v4i64),
lib/Target/X86/X86ISelLowering.cpp:    OpLo = DAG.getVectorShuffle(MVT::v16i8, DL, OpLo, Undef, ShufMask1);
lib/Target/X86/X86ISelLowering.cpp:    OpHi = DAG.getVectorShuffle(MVT::v16i8, DL, OpHi, Undef, ShufMask1);
lib/Target/X86/X86ISelLowering.cpp:    SDValue res = DAG.getVectorShuffle(MVT::v4i32, DL, OpLo, OpHi, ShufMask2);
lib/Target/X86/X86ISelLowering.cpp:  // Handle truncation of V256 to V128 using shuffles.
lib/Target/X86/X86ISelLowering.cpp:  // Prepare truncation shuffle mask
lib/Target/X86/X86ISelLowering.cpp:  SDValue V = DAG.getVectorShuffle(NVT, DL,
lib/Target/X86/X86ISelLowering.cpp:      SDValue EQHi = DAG.getVectorShuffle(MVT::v4i32, dl, EQ, EQ, MaskHi);
lib/Target/X86/X86ISelLowering.cpp:      SDValue GTLo = DAG.getVectorShuffle(MVT::v4i32, dl, GT, GT, MaskLo);
lib/Target/X86/X86ISelLowering.cpp:      SDValue GTHi = DAG.getVectorShuffle(MVT::v4i32, dl, GT, GT, MaskHi);
lib/Target/X86/X86ISelLowering.cpp:      SDValue Shuf = DAG.getVectorShuffle(MVT::v4i32, dl, Result, Result, Mask);
lib/Target/X86/X86ISelLowering.cpp:  // for v4i32 the shuffle mask will be { 0, 1, -1, -1} {2, 3, -1, -1}
lib/Target/X86/X86ISelLowering.cpp:  SDValue OpLo = DAG.getVectorShuffle(InVT, dl, In, Undef, &ShufMask1[0]);
lib/Target/X86/X86ISelLowering.cpp:  SDValue OpHi = DAG.getVectorShuffle(InVT, dl, In, Undef, &ShufMask2[0]);
lib/Target/X86/X86ISelLowering.cpp:  // Lower v4i32 mul as 2x shuffle, 2x pmuludq, 2x shuffle.
lib/Target/X86/X86ISelLowering.cpp:    SDValue Aodds = DAG.getVectorShuffle(VT, dl, A, A, UnpackMask);
lib/Target/X86/X86ISelLowering.cpp:    SDValue Bodds = DAG.getVectorShuffle(VT, dl, B, B, UnpackMask);
lib/Target/X86/X86ISelLowering.cpp:    // Merge the two vectors back together with a shuffle. This expands into 2
lib/Target/X86/X86ISelLowering.cpp:    // shuffles.
lib/Target/X86/X86ISelLowering.cpp:    return DAG.getVectorShuffle(VT, dl, Evens, Odds, ShufMask);
lib/Target/X86/X86ISelLowering.cpp:               cast<ShuffleVectorSDNode>(Amt)->isSplat()) {
lib/Target/X86/X86ISelLowering.cpp:               cast<ShuffleVectorSDNode>(Amt)->getSplatIndex();
lib/Target/X86/X86ISelLowering.cpp:/// isShuffleMaskLegal - Targets can use this to indicate that they only
lib/Target/X86/X86ISelLowering.cpp:X86TargetLowering::isShuffleMaskLegal(const SmallVectorImpl<int> &M,
lib/Target/X86/X86ISelLowering.cpp:          ShuffleVectorSDNode::isSplatMask(&M[0], VT) ||
lib/Target/X86/X86ISelLowering.cpp:/// isShuffleHigh128VectorInsertLow - Checks whether the shuffle node is the
lib/Target/X86/X86ISelLowering.cpp:static bool isShuffleHigh128VectorInsertLow(ShuffleVectorSDNode *SVOp) {
lib/Target/X86/X86ISelLowering.cpp:  // vector_shuffle <4, 5, 6, 7, u, u, u, u> or <2, 3, u, u>
lib/Target/X86/X86ISelLowering.cpp:/// isShuffleLow128VectorInsertHigh - Checks whether the shuffle node is the
lib/Target/X86/X86ISelLowering.cpp:static bool isShuffleLow128VectorInsertHigh(ShuffleVectorSDNode *SVOp) {
lib/Target/X86/X86ISelLowering.cpp:  // vector_shuffle <u, u, u, u, 0, 1, 2, 3> or <u, u, 0, 1>
lib/Target/X86/X86ISelLowering.cpp:/// PerformShuffleCombine256 - Performs shuffle combines for 256-bit vectors.
lib/Target/X86/X86ISelLowering.cpp:static SDValue PerformShuffleCombine256(SDNode *N, SelectionDAG &DAG,
lib/Target/X86/X86ISelLowering.cpp:  ShuffleVectorSDNode *SVOp = cast<ShuffleVectorSDNode>(N);
lib/Target/X86/X86ISelLowering.cpp:    // To match the shuffle mask, the first half of the mask should
lib/Target/X86/X86ISelLowering.cpp:  // Combine some shuffles into subvector extracts and inserts:
lib/Target/X86/X86ISelLowering.cpp:  // vector_shuffle <4, 5, 6, 7, u, u, u, u> or <2, 3, u, u>
lib/Target/X86/X86ISelLowering.cpp:  if (isShuffleHigh128VectorInsertLow(SVOp)) {
lib/Target/X86/X86ISelLowering.cpp:  // vector_shuffle <u, u, u, u, 0, 1, 2, 3> or <u, u, 0, 1>
lib/Target/X86/X86ISelLowering.cpp:  if (isShuffleLow128VectorInsertHigh(SVOp)) {
lib/Target/X86/X86ISelLowering.cpp:/// PerformShuffleCombine - Performs several different shuffle combines.
lib/Target/X86/X86ISelLowering.cpp:static SDValue PerformShuffleCombine(SDNode *N, SelectionDAG &DAG,
lib/Target/X86/X86ISelLowering.cpp:  // Combine 256-bit vector shuffles. This is only profitable when in AVX mode
lib/Target/X86/X86ISelLowering.cpp:    return PerformShuffleCombine256(N, DAG, DCI, Subtarget);
lib/Target/X86/X86ISelLowering.cpp:  // Combine a vector_shuffle that is equal to build_vector load1, load2, load3,
lib/Target/X86/X86ISelLowering.cpp:    Elts.push_back(getShuffleScalarElt(N, i, DAG, 0));
lib/Target/X86/X86ISelLowering.cpp:/// a sequence of vector shuffle operations.
lib/Target/X86/X86ISelLowering.cpp:/// XFormVExtractWithShuffleIntoLoad - Check if a vector extract from a target
lib/Target/X86/X86ISelLowering.cpp:/// specific shuffle of a load can be folded into a single element load.
lib/Target/X86/X86ISelLowering.cpp:/// shuffles have been customed lowered so we need to handle those here.
lib/Target/X86/X86ISelLowering.cpp:static SDValue XFormVExtractWithShuffleIntoLoad(SDNode *N, SelectionDAG &DAG,
lib/Target/X86/X86ISelLowering.cpp:  bool HasShuffleIntoBitcast = false;
lib/Target/X86/X86ISelLowering.cpp:    HasShuffleIntoBitcast = true;
lib/Target/X86/X86ISelLowering.cpp:  if (!isTargetShuffle(InVec.getOpcode()))
lib/Target/X86/X86ISelLowering.cpp:  SmallVector<int, 16> ShuffleMask;
lib/Target/X86/X86ISelLowering.cpp:  bool UnaryShuffle;
lib/Target/X86/X86ISelLowering.cpp:  if (!getTargetShuffleMask(InVec.getNode(), VT.getSimpleVT(), ShuffleMask,
lib/Target/X86/X86ISelLowering.cpp:                            UnaryShuffle))
lib/Target/X86/X86ISelLowering.cpp:  int Idx = (Elt > (int)NumElems) ? -1 : ShuffleMask[Elt];
lib/Target/X86/X86ISelLowering.cpp:  // If inputs to shuffle are the same for both ops, then allow 2 uses
lib/Target/X86/X86ISelLowering.cpp:  if (HasShuffleIntoBitcast) {
lib/Target/X86/X86ISelLowering.cpp:    // If there's a bitcast before the shuffle, check if the load type and
lib/Target/X86/X86ISelLowering.cpp:  // All checks match so transform back to vector_shuffle so that DAG combiner
lib/Target/X86/X86ISelLowering.cpp:  // Create shuffle node taking into account the case that its a unary shuffle
lib/Target/X86/X86ISelLowering.cpp:  SDValue Shuffle = (UnaryShuffle) ? DAG.getUNDEF(VT) : InVec.getOperand(1);
lib/Target/X86/X86ISelLowering.cpp:  Shuffle = DAG.getVectorShuffle(InVec.getValueType(), dl,
lib/Target/X86/X86ISelLowering.cpp:                                 InVec.getOperand(0), Shuffle,
lib/Target/X86/X86ISelLowering.cpp:                                 &ShuffleMask[0]);
lib/Target/X86/X86ISelLowering.cpp:  Shuffle = DAG.getNode(ISD::BITCAST, dl, VT, Shuffle);
lib/Target/X86/X86ISelLowering.cpp:  return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, dl, N->getValueType(0), Shuffle,
lib/Target/X86/X86ISelLowering.cpp:/// generation and convert it from being a bunch of shuffles and extracts
lib/Target/X86/X86ISelLowering.cpp:  SDValue NewOp = XFormVExtractWithShuffleIntoLoad(N, DAG, DCI);
lib/Target/X86/X86ISelLowering.cpp:          return getTargetShuffleNode(X86ISD::MOVSS, DL, VT, A, B, DAG);
lib/Target/X86/X86ISelLowering.cpp:        return getTargetShuffleNode(X86ISD::MOVSD, DL, VT, A, B, DAG);
lib/Target/X86/X86ISelLowering.cpp:          SDValue Select = getTargetShuffleNode(X86ISD::MOVSD, DL, NVT, NewA,
lib/Target/X86/X86ISelLowering.cpp:  // shuffle. If SSSE3 is not available we may emit an illegal shuffle but the
lib/Target/X86/X86ISelLowering.cpp:  // emit a shuffle and a arithmetic shift.
lib/Target/X86/X86ISelLowering.cpp:  // during the shuffle phase or after the shuffle.
lib/Target/X86/X86ISelLowering.cpp:    // We can't shuffle using an illegal type.
lib/Target/X86/X86ISelLowering.cpp:      // Otherwise we'll shuffle the small elements in the high bits of the
lib/Target/X86/X86ISelLowering.cpp:      SmallVector<int, 8> ShuffleVec(NumElems * SizeRatio, -1);
lib/Target/X86/X86ISelLowering.cpp:        ShuffleVec[i*SizeRatio + SizeRatio-1] = i;
lib/Target/X86/X86ISelLowering.cpp:      SDValue Shuff = DAG.getVectorShuffle(WideVecVT, dl, SlicedVec,
lib/Target/X86/X86ISelLowering.cpp:                                           &ShuffleVec[0]);
lib/Target/X86/X86ISelLowering.cpp:    SmallVector<int, 8> ShuffleVec(NumElems * SizeRatio, -1);
lib/Target/X86/X86ISelLowering.cpp:      ShuffleVec[i*SizeRatio] = i;
lib/Target/X86/X86ISelLowering.cpp:    SDValue Shuff = DAG.getVectorShuffle(WideVecVT, dl, SlicedVec,
lib/Target/X86/X86ISelLowering.cpp:                                         &ShuffleVec[0]);
lib/Target/X86/X86ISelLowering.cpp:  // Optimize trunc store (of multiple scalars) to shuffle and store.
lib/Target/X86/X86ISelLowering.cpp:    // Create a type on which we perform the shuffle
lib/Target/X86/X86ISelLowering.cpp:    SmallVector<int, 8> ShuffleVec(NumElems * SizeRatio, -1);
lib/Target/X86/X86ISelLowering.cpp:      ShuffleVec[i] = i * SizeRatio;
lib/Target/X86/X86ISelLowering.cpp:    // Can't shuffle using an illegal type.
lib/Target/X86/X86ISelLowering.cpp:    SDValue Shuff = DAG.getVectorShuffle(WideVecVT, dl, WideVec,
lib/Target/X86/X86ISelLowering.cpp:                                         &ShuffleVec[0]);
lib/Target/X86/X86ISelLowering.cpp:  // At least one of the operands should be a vector shuffle.
lib/Target/X86/X86ISelLowering.cpp:  // If LHS is not a shuffle then pretend it is the shuffle
lib/Target/X86/X86ISelLowering.cpp:    ArrayRef<int> Mask = cast<ShuffleVectorSDNode>(LHS.getNode())->getMask();
lib/Target/X86/X86ISelLowering.cpp:    ArrayRef<int> Mask = cast<ShuffleVectorSDNode>(RHS.getNode())->getMask();
lib/Target/X86/X86ISelLowering.cpp:  // Check that the shuffles are both shuffling the same vectors.
lib/Target/X86/X86ISelLowering.cpp:    CommuteVectorShuffleMask(RMask, NumElts);
lib/Target/X86/X86ISelLowering.cpp:  // Try to synthesize horizontal adds from adds of shuffles.
lib/Target/X86/X86ISelLowering.cpp:  // Try to synthesize horizontal subs from subs of shuffles.
lib/Target/X86/X86ISelLowering.cpp:  // Try to synthesize horizontal adds from adds of shuffles.
lib/Target/X86/X86ISelLowering.cpp:  // Try to synthesize horizontal adds from adds of shuffles.
lib/Target/X86/X86ISelLowering.cpp:  case X86ISD::SHUFP:       // Handle all target specific shuffles
lib/Target/X86/X86ISelLowering.cpp:  case ISD::VECTOR_SHUFFLE: return PerformShuffleCombine(N, DAG, DCI,Subtarget);
lib/Target/X86/X86ISelLowering.h:      /// PSHUFB - Shuffle 16 8-bit values within a vector.
lib/Target/X86/X86ISelLowering.h:      // Several flavors of instructions with vector shuffle behaviors.
lib/Target/X86/X86ISelLowering.h:    /// isShuffleMaskLegal - Targets can use this to indicate that they only
lib/Target/X86/X86ISelLowering.h:    virtual bool isShuffleMaskLegal(const SmallVectorImpl<int> &Mask,
lib/Target/X86/X86ISelLowering.h:    /// isVectorClearMaskLegal - Similar to isShuffleMaskLegal. This is
lib/Target/X86/X86InstrAVX512.td:  // Shuffle with VMOVSS
lib/Target/X86/X86InstrAVX512.td:  // Shuffle with VMOVSD
lib/Target/X86/X86InstrAVX512.td:                   EVEX_4V, Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrAVX512.td:                   EVEX_4V, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrFragmentsSIMD.td:// Specific shuffle nodes - At some point ISD::VECTOR_SHUFFLE will always get
lib/Target/X86/X86InstrInfo.cpp:// FIXME: Some shuffle and unpack instructions have equivalents in different
lib/Target/X86/X86InstrMMX.td:let Sched = WriteShuffle in {
lib/Target/X86/X86InstrMMX.td:// -- Shuffle Instructions
lib/Target/X86/X86InstrMMX.td:                          IIC_MMX_PSHUF>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrMMX.td:                          IIC_MMX_PSHUF>, Sched<[WriteShuffleLd]>;
lib/Target/X86/X86InstrMMX.td:                       IIC_MMX_PEXTR>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrMMX.td:                      IIC_MMX_PINSRW>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrMMX.td:                     IIC_MMX_PINSRW>, Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrMMX.td:let SchedRW = [WriteShuffle] in {
lib/Target/X86/X86InstrSSE.td:let Sched = WriteShuffle in
lib/Target/X86/X86InstrSSE.td:let Sched = WriteShuffle in
lib/Target/X86/X86InstrSSE.td:              IIC_SSE_MOV_S_RR>, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:                  [], IIC_SSE_MOV_S_RR>, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:  // Shuffle with VMOVSS
lib/Target/X86/X86InstrSSE.td:  // Shuffle with VMOVSD
lib/Target/X86/X86InstrSSE.td:  // Shuffle with MOVSS
lib/Target/X86/X86InstrSSE.td:  // Shuffle with MOVSD
lib/Target/X86/X86InstrSSE.td:           Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:    SchedRW = [WriteFShuffle] in {
lib/Target/X86/X86InstrSSE.td:     Sched<[WriteFShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:     Sched<[WriteFShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:  // Shuffle with VMOVLPS
lib/Target/X86/X86InstrSSE.td:  // Shuffle with VMOVLPD
lib/Target/X86/X86InstrSSE.td:  // (store (vector_shuffle (load addr), v2, <4, 5, 2, 3>), addr) using MOVLPS
lib/Target/X86/X86InstrSSE.td:  // Shuffle with MOVLPS
lib/Target/X86/X86InstrSSE.td:  // Shuffle with MOVLPD
lib/Target/X86/X86InstrSSE.td:                      VEX_4V, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:                      VEX_4V, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:                        IIC_SSE_MOV_LH>, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:                        IIC_SSE_MOV_LH>, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:// SSE 1 & 2 - Shuffle Instructions
lib/Target/X86/X86InstrSSE.td:/// sse12_shuffle - sse 1 & 2 fp shuffle instructions
lib/Target/X86/X86InstrSSE.td:multiclass sse12_shuffle<RegisterClass RC, X86MemOperand x86memop,
lib/Target/X86/X86InstrSSE.td:            Sched<[WriteFShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:              Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:defm VSHUFPS  : sse12_shuffle<VR128, f128mem, v4f32,
lib/Target/X86/X86InstrSSE.td:defm VSHUFPSY : sse12_shuffle<VR256, f256mem, v8f32,
lib/Target/X86/X86InstrSSE.td:defm VSHUFPD  : sse12_shuffle<VR128, f128mem, v2f64,
lib/Target/X86/X86InstrSSE.td:defm VSHUFPDY : sse12_shuffle<VR256, f256mem, v4f64,
lib/Target/X86/X86InstrSSE.td:  defm SHUFPS : sse12_shuffle<VR128, f128mem, v4f32,
lib/Target/X86/X86InstrSSE.td:  defm SHUFPD : sse12_shuffle<VR128, f128mem, v2f64,
lib/Target/X86/X86InstrSSE.td:                           IIC_SSE_UNPCK, d>, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:             Sched<[WriteFShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:// SSE2 - Packed Integer Shuffle Instructions
lib/Target/X86/X86InstrSSE.td:multiclass sse2_pshuffle<string OpcodeStr, ValueType vt128, ValueType vt256,
lib/Target/X86/X86InstrSSE.td:                      IIC_SSE_PSHUF_RI>, VEX, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:                  Sched<[WriteShuffleLd]>;
lib/Target/X86/X86InstrSSE.td:                       IIC_SSE_PSHUF_RI>, VEX, VEX_L, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:                   Sched<[WriteShuffleLd]>;
lib/Target/X86/X86InstrSSE.td:                IIC_SSE_PSHUF_RI>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:           Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:defm PSHUFD  : sse2_pshuffle<"pshufd", v4i32, v8i32, X86PShufd>, PD;
lib/Target/X86/X86InstrSSE.td:defm PSHUFHW : sse2_pshuffle<"pshufhw", v8i16, v16i16, X86PShufhw>, XS;
lib/Target/X86/X86InstrSSE.td:defm PSHUFLW : sse2_pshuffle<"pshuflw", v8i16, v16i16, X86PShuflw>, XD;
lib/Target/X86/X86InstrSSE.td:      IIC_SSE_UNPCK>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:       IIC_SSE_PINSRW>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:       Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:                Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:               Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:                      IIC_SSE_MOV_LH>, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:                    [], IIC_SSE_MOV_LH>, Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:                    Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:let Sched = WriteShuffle in
lib/Target/X86/X86InstrSSE.td:      [], IIC_SSE_PALIGNRR>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:      [], IIC_SSE_PALIGNRM>, Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:      []>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:      []>, Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:                                        WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                        WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                        WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                        WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                        WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                        WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                       int_x86_avx2_pmovsxbd, WriteShuffle>,
lib/Target/X86/X86InstrSSE.td:                                       int_x86_avx2_pmovsxwq, WriteShuffle>,
lib/Target/X86/X86InstrSSE.td:                                       int_x86_avx2_pmovzxbd, WriteShuffle>,
lib/Target/X86/X86InstrSSE.td:                                       int_x86_avx2_pmovzxwq, WriteShuffle>,
lib/Target/X86/X86InstrSSE.td:                                     WriteShuffle>, VEX;
lib/Target/X86/X86InstrSSE.td:                                     WriteShuffle>, VEX;
lib/Target/X86/X86InstrSSE.td:                                       WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                       WriteShuffle>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                      WriteShuffle>;
lib/Target/X86/X86InstrSSE.td:                                      WriteShuffle>;
lib/Target/X86/X86InstrSSE.td:                  Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:      SchedRW = [WriteShuffleLd, WriteRMW] in
lib/Target/X86/X86InstrSSE.td:                   []>, Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:      SchedRW = [WriteShuffleLd, WriteRMW] in
lib/Target/X86/X86InstrSSE.td:                  Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:  let SchedRW = [WriteShuffleLd, WriteRMW] in
lib/Target/X86/X86InstrSSE.td:                  Sched<[WriteShuffle]>, REX_W;
lib/Target/X86/X86InstrSSE.td:  let SchedRW = [WriteShuffleLd, WriteRMW] in
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:                   imm:$src3))]>, Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:                          imm:$src3)))]>, Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteShuffle]>;
lib/Target/X86/X86InstrSSE.td:                          imm:$src3)))]>, Sched<[WriteShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:      Sched<[WriteFShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:                                        int_x86_avx2_packusdw, WriteShuffle>,
lib/Target/X86/X86InstrSSE.td:                                      WriteFShuffleLd>, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                    WriteFShuffleLd>, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                   WriteFShuffleLd>, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                           WriteFShuffle>;
lib/Target/X86/X86InstrSSE.td:                                      WriteFShuffle256>, VEX_L;
lib/Target/X86/X86InstrSSE.td:                                      WriteFShuffle256>, VEX_L;
lib/Target/X86/X86InstrSSE.td:          []>, Sched<[WriteFShuffle]>, VEX_4V, VEX_L;
lib/Target/X86/X86InstrSSE.td:          []>, Sched<[WriteFShuffleLd, ReadAfterLd]>, VEX_4V, VEX_L;
lib/Target/X86/X86InstrSSE.td:          []>, Sched<[WriteFShuffle]>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:             Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:             Sched<[WriteFShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:             Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:             Sched<[WriteFShuffleLd]>;
lib/Target/X86/X86InstrSSE.td:          Sched<[WriteFShuffle]>;
lib/Target/X86/X86InstrSSE.td:          Sched<[WriteFShuffleLd, ReadAfterLd]>;
lib/Target/X86/X86InstrSSE.td:                  Sched<[WriteShuffle]>, VEX;
lib/Target/X86/X86InstrSSE.td:                   Sched<[WriteShuffle256]>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                   Sched<[WriteFShuffle256]>, VEX_4V, VEX_L;
lib/Target/X86/X86InstrSSE.td:                   Sched<[WriteFShuffle256Ld, ReadAfterLd]>, VEX_4V, VEX_L;
lib/Target/X86/X86InstrSSE.td:                     Sched<[WriteShuffle256]>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                     Sched<[WriteShuffle256Ld, ReadAfterLd]>, VEX, VEX_L;
lib/Target/X86/X86InstrSSE.td:                            (i8 imm:$src3))))]>, Sched<[WriteShuffle256]>,
lib/Target/X86/X86InstrSSE.td:          Sched<[WriteShuffle256Ld, ReadAfterLd]>, VEX_4V, VEX_L;
lib/Target/X86/X86InstrSSE.td:          []>, Sched<[WriteShuffle256]>, VEX_4V, VEX_L;
lib/Target/X86/X86InstrSSE.td:          []>, Sched<[WriteShuffle256Ld, ReadAfterLd]>, VEX_4V, VEX_L;
lib/Target/X86/X86InstrSSE.td:          Sched<[WriteShuffle256]>, VEX, VEX_L;
lib/Target/X86/X86SchedHaswell.td:defm : HWWriteResPair<WriteFShuffle,  HWPort5,  1>;
lib/Target/X86/X86SchedHaswell.td:defm : HWWriteResPair<WriteFShuffle256,  HWPort5,  3>;
lib/Target/X86/X86SchedHaswell.td:defm : HWWriteResPair<WriteShuffle,  HWPort5,  1>;
lib/Target/X86/X86SchedHaswell.td:defm : HWWriteResPair<WriteShuffle256,  HWPort5,  3>;
lib/Target/X86/X86SchedSandyBridge.td:defm : SBWriteResPair<WriteFShuffle,  SBPort5,  1>;
lib/Target/X86/X86SchedSandyBridge.td:defm : SBWriteResPair<WriteShuffle,  SBPort15,  1>;
lib/Target/X86/X86SchedSandyBridge.td:defm : SBWriteResPair<WriteFShuffle256, SBPort0,  1>;
lib/Target/X86/X86SchedSandyBridge.td:defm : SBWriteResPair<WriteShuffle256, SBPort0,  1>;
lib/Target/X86/X86Schedule.td:defm WriteFShuffle  : X86SchedWritePair; // Floating point vector shuffles.
lib/Target/X86/X86Schedule.td:defm WriteShuffle  : X86SchedWritePair; // Vector shuffles.
lib/Target/X86/X86Schedule.td:defm WriteFShuffle256 : X86SchedWritePair; // Fp 256-bit width vector shuffles.
lib/Target/X86/X86Schedule.td:defm WriteShuffle256 : X86SchedWritePair; // 256-bit width vector shuffles.
lib/Target/X86/X86ScheduleAtom.td:                      // SIMD/FP: SIMD ALU, Shuffle,SIMD/FP multiply, divide
lib/Target/X86/X86TargetTransformInfo.cpp:  virtual unsigned getShuffleCost(ShuffleKind Kind, Type *Tp,
lib/Target/X86/X86TargetTransformInfo.cpp:      // lowered into a sequence of shuffles and 2 x pmuludq.
lib/Target/X86/X86TargetTransformInfo.cpp:  // Special lowering of v4i32 mul on sse2, sse3: Lower v4i32 mul as 2x shuffle,
lib/Target/X86/X86TargetTransformInfo.cpp:  // 2x pmuludq, 2x shuffle.
lib/Target/X86/X86TargetTransformInfo.cpp:unsigned X86TTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
lib/Target/X86/X86TargetTransformInfo.cpp:  // We only estimate the cost of reverse shuffles.
lib/Target/X86/X86TargetTransformInfo.cpp:    return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp);
lib/Transforms/InstCombine/InstCombine.h:  Instruction *visitShuffleVectorInst(ShuffleVectorInst &SVI);
lib/Transforms/InstCombine/InstCombineCalls.cpp:    // Turn vperm(V1,V2,mask) -> shuffle(V1,V2,mask) if mask is a constant.
lib/Transforms/InstCombine/InstCombineCasts.cpp:/// replace it with a shuffle (and vector/vector bitcast) if possible.
lib/Transforms/InstCombine/InstCombineCasts.cpp:  // Now that the element types match, get the shuffle mask and RHS of the
lib/Transforms/InstCombine/InstCombineCasts.cpp:  // shuffle to use, which depends on whether we're increasing or decreasing the
lib/Transforms/InstCombine/InstCombineCasts.cpp:  SmallVector<uint32_t, 16> ShuffleMask;
lib/Transforms/InstCombine/InstCombineCasts.cpp:    // If we're shrinking the number of elements, just shuffle in the low
lib/Transforms/InstCombine/InstCombineCasts.cpp:    // elements from the input and use undef as the second shuffle input.
lib/Transforms/InstCombine/InstCombineCasts.cpp:      ShuffleMask.push_back(i);
lib/Transforms/InstCombine/InstCombineCasts.cpp:    // If we're increasing the number of elements, shuffle in all of the
lib/Transforms/InstCombine/InstCombineCasts.cpp:      ShuffleMask.push_back(i);
lib/Transforms/InstCombine/InstCombineCasts.cpp:      ShuffleMask.push_back(SrcElts);
lib/Transforms/InstCombine/InstCombineCasts.cpp:  return new ShuffleVectorInst(InVal, V2,
lib/Transforms/InstCombine/InstCombineCasts.cpp:                                                       ShuffleMask));
lib/Transforms/InstCombine/InstCombineCasts.cpp:      // the casts with a shuffle and (potentially) a bitcast.
lib/Transforms/InstCombine/InstCombineCasts.cpp:  if (ShuffleVectorInst *SVI = dyn_cast<ShuffleVectorInst>(Src)) {
lib/Transforms/InstCombine/InstCombineCasts.cpp:    // Okay, we have (bitcast (shuffle ..)).  Check to see if this is
lib/Transforms/InstCombine/InstCombineCasts.cpp:      // evaluating the shuffle in the casted destination's type will allow
lib/Transforms/InstCombine/InstCombineCasts.cpp:        // Return a new shuffle vector.  Use the same element ID's, as we
lib/Transforms/InstCombine/InstCombineCasts.cpp:        return new ShuffleVectorInst(LHS, RHS, SVI->getOperand(2));
lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp:  case Instruction::ShuffleVector: {
lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp:    ShuffleVectorInst *Shuffle = cast<ShuffleVectorInst>(I);
lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp:      cast<VectorType>(Shuffle->getOperand(0)->getType())->getNumElements();
lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp:        unsigned MaskVal = Shuffle->getMaskValue(i);
lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp:                 "shufflevector mask index out of range!");
lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp:      unsigned MaskVal = Shuffle->getMaskValue(i);
lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp:                                          Shuffle->getMaskValue(i)));
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:// ShuffleVector.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (ShuffleVectorInst *SVI = dyn_cast<ShuffleVectorInst>(V)) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    } else if (ShuffleVectorInst *SVI = dyn_cast<ShuffleVectorInst>(I)) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      // If this is extracting an element from a shufflevector, figure out where
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:/// CollectSingleShuffleElements - If V is a shuffle of values that ONLY returns
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:/// elements from either LHS or RHS, return the shuffle mask and true.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:static bool CollectSingleShuffleElements(Value *V, Value *LHS, Value *RHS,
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:         "Invalid CollectSingleShuffleElements");
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      if (CollectSingleShuffleElements(VecOp, LHS, RHS, Mask)) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:          if (CollectSingleShuffleElements(VecOp, LHS, RHS, Mask)) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // TODO: Handle shufflevector here!
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:/// CollectShuffleElements - We are building a shuffle of V, using RHS as the
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:/// RHS of the shuffle instruction, if it is not null.  Return a shuffle mask
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:/// that computes V and the LHS value of the shuffle.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:static Value *CollectShuffleElements(Value *V, SmallVectorImpl<Constant*> &Mask,
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:         "Invalid shuffle!");
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:        // otherwise we'd end up with a shuffle of three inputs.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:          Value *V = CollectShuffleElements(VecOp, Mask, RHS);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:          Value *V = CollectShuffleElements(EI->getOperand(0), Mask, RHS);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:        // vectors, return the vector and the effective shuffle.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:        if (CollectSingleShuffleElements(IEI, EI->getOperand(0), RHS, Mask))
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // TODO: Handle shufflevector here!
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // indexes are constant, try to turn this into a shufflevector operation.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      // (and any insertelements it points to), into one big shuffle.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:        Value *LHS = CollectShuffleElements(&IE, Mask, RHS);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:        // We now have a shuffle of LHS, RHS, Mask.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:        return new ShuffleVectorInst(LHS, RHS, ConstantVector::get(Mask));
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:/// elements were shuffled in a different order.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:static bool CanEvaluateShuffled(Value *V, ArrayRef<int> Mask,
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:        if (!CanEvaluateShuffled(I->getOperand(i), Mask, Depth-1))
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      return CanEvaluateShuffled(I->getOperand(0), Mask, Depth-1);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    return ConstantExpr::getShuffleVector(C, UndefValue::get(C->getType()),
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      // by CanEvaluateShuffled.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:Instruction *InstCombiner::visitShuffleVectorInst(ShuffleVectorInst &SVI) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  SmallVector<int, 16> Mask = SVI.getShuffleMask();
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // Undefined shuffle mask -> undefined value.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // Canonicalize shuffle(x    ,x,mask) -> shuffle(x, undef,mask')
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // Canonicalize shuffle(undef,x,mask) -> shuffle(x, undef,mask').
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      // shuffle(undef,undef,mask) -> undef.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    // Analyze the shuffle, are the LHS or RHS and identity shuffles?
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      // Is this an identity shuffle of the LHS value?
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      // Is this an identity shuffle of the RHS value?
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    // Eliminate identity shuffles.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (isa<UndefValue>(RHS) && CanEvaluateShuffled(LHS, Mask)) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // If the LHS is a shufflevector itself, see if we can combine it with this
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // one without producing an unusual shuffle.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // x1=shuffle(v1,v2,mask1)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(x1,undef,mask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(v1,undef,newMask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // x1=shuffle(v1,undef,mask1)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(x1,x2,mask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(v1,x2,newMask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // x2=shuffle(v2,undef,mask2)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(x1,x2,mask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(x1,v2,newMask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // x1=shuffle(v1,undef,mask1)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // x2=shuffle(v2,undef,mask2)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(x1,x2,mask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  //  x=shuffle(v1,v2,newMask)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // we are absolutely afraid of producing a shuffle mask not in the input
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // shuffle into two specific shuffles: it may produce worse code.  As such,
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // we only merge two shuffles if the result is either a splat or one of the
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // input shuffle masks.  In this case, merging the shuffles just removes
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  ShuffleVectorInst* LHSShuffle = dyn_cast<ShuffleVectorInst>(LHS);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  ShuffleVectorInst* RHSShuffle = dyn_cast<ShuffleVectorInst>(RHS);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (LHSShuffle)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    if (!isa<UndefValue>(LHSShuffle->getOperand(1)) && !isa<UndefValue>(RHS))
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      LHSShuffle = NULL;
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (RHSShuffle)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    if (!isa<UndefValue>(RHSShuffle->getOperand(1)))
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:      RHSShuffle = NULL;
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (!LHSShuffle && !RHSShuffle)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (LHSShuffle) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    LHSOp0 = LHSShuffle->getOperand(0);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    LHSOp1 = LHSShuffle->getOperand(1);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (RHSShuffle) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    RHSOp0 = RHSShuffle->getOperand(0);
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (LHSShuffle) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (RHSShuffle && RHSOp0Width == LHSWidth) {
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    LHSMask = LHSShuffle->getShuffleMask();
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  if (RHSShuffle && newRHS != RHS)
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    RHSMask = RHSShuffle->getShuffleMask();
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // Create a new mask for the new ShuffleVectorInst so that the new
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // ShuffleVectorInst is equivalent to the original one.
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:          assert(isa<UndefValue>(RHSShuffle->getOperand(1))
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:  // If the result mask is equal to one of the original shuffle masks,
lib/Transforms/InstCombine/InstCombineVectorOps.cpp:    return new ShuffleVectorInst(newLHS, newRHS, ConstantVector::get(Elts));
lib/Transforms/Instrumentation/DataFlowSanitizer.cpp:  void visitShuffleVectorInst(ShuffleVectorInst &I);
lib/Transforms/Instrumentation/DataFlowSanitizer.cpp:void DFSanVisitor::visitShuffleVectorInst(ShuffleVectorInst &I) {
lib/Transforms/Instrumentation/MemorySanitizer.cpp:  void visitShuffleVectorInst(ShuffleVectorInst &I) {
lib/Transforms/Instrumentation/MemorySanitizer.cpp:    setShadow(&I, IRB.CreateShuffleVector(getShadow(&I, 0), getShadow(&I, 1),
lib/Transforms/ObjCARC/ObjCARCUtil.cpp:    case Instruction::ShuffleVector:
lib/Transforms/Scalar/EarlyCSE.cpp:             isa<InsertElementInst>(Inst) || isa<ShuffleVectorInst>(Inst) ||
lib/Transforms/Scalar/EarlyCSE.cpp:          isa<ShuffleVectorInst>(Inst)) && "Invalid/unknown instruction");
lib/Transforms/Scalar/GVN.cpp:    case Instruction::ShuffleVector:
lib/Transforms/Scalar/LICM.cpp:      !isa<ShuffleVectorInst>(I) && !isa<ExtractValueInst>(I) &&
lib/Transforms/Scalar/SCCP.cpp:  void visitShuffleVectorInst(ShuffleVectorInst &I);
lib/Transforms/Scalar/SCCP.cpp:void SCCPSolver::visitShuffleVectorInst(ShuffleVectorInst &I) {
lib/Transforms/Scalar/SCCP.cpp:    markConstant(&I, ConstantExpr::getShuffleVector(V1, V2, Mask));
lib/Transforms/Scalar/SROA.cpp:static cl::opt<bool> SROARandomShuffleSlices("sroa-random-shuffle-slices",
lib/Transforms/Scalar/SROA.cpp:  if (SROARandomShuffleSlices) {
lib/Transforms/Scalar/SROA.cpp:    std::shuffle(Slices.begin(), Slices.end(), MT);
lib/Transforms/Scalar/SROA.cpp:  V = IRB.CreateShuffleVector(V, UndefValue::get(V->getType()),
lib/Transforms/Scalar/SROA.cpp:  DEBUG(dbgs() << "     shuffle: " << *V << "\n");
lib/Transforms/Scalar/SROA.cpp:  // use a shuffle vector to widen it with undef elements, and then
lib/Transforms/Scalar/SROA.cpp:  // a second shuffle vector to select between the loaded vector and the
lib/Transforms/Scalar/SROA.cpp:  V = IRB.CreateShuffleVector(V, UndefValue::get(V->getType()),
lib/Transforms/Scalar/SROA.cpp:  DEBUG(dbgs() << "    shuffle: " << *V << "\n");
lib/Transforms/Scalar/Scalarizer.cpp:  bool visitShuffleVectorInst(ShuffleVectorInst &);
lib/Transforms/Scalar/Scalarizer.cpp:bool Scalarizer::visitShuffleVectorInst(ShuffleVectorInst &SVI) {
lib/Transforms/Vectorize/BBVectorize.cpp:    void fillNewShuffleMask(LLVMContext& Context, Instruction *J,
lib/Transforms/Vectorize/BBVectorize.cpp:    Value *getReplacementShuffleMask(LLVMContext& Context, Instruction *I,
lib/Transforms/Vectorize/BBVectorize.cpp:      } else if (ShuffleVectorInst *SI = dyn_cast<ShuffleVectorInst>(I)) {
lib/Transforms/Vectorize/BBVectorize.cpp:      case Instruction::ShuffleVector:
lib/Transforms/Vectorize/BBVectorize.cpp:    } else if (!(I->isBinaryOp() || isa<ShuffleVectorInst>(I) ||
lib/Transforms/Vectorize/BBVectorize.cpp:      // instructions (because combining these often results in a shuffle),
lib/Transforms/Vectorize/BBVectorize.cpp:        // shuffle chains.
lib/Transforms/Vectorize/BBVectorize.cpp:          if (!isa<ShuffleVectorInst>(S->first) &&
lib/Transforms/Vectorize/BBVectorize.cpp:          // the cost of shuffles.
lib/Transforms/Vectorize/BBVectorize.cpp:                int ESContrib = (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:                    ESContrib = std::min(ESContrib, (int) TTI->getShuffleCost(
lib/Transforms/Vectorize/BBVectorize.cpp:                    ESContrib = std::min(ESContrib, (int) TTI->getShuffleCost(
lib/Transforms/Vectorize/BBVectorize.cpp:          // to shuffles, inserts or extracts can be merged, and so contribute
lib/Transforms/Vectorize/BBVectorize.cpp:              if (ShuffleVectorInst *SI = dyn_cast<ShuffleVectorInst>(*I)) {
lib/Transforms/Vectorize/BBVectorize.cpp:                // Shuffle can be folded if it has no other input
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib = (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib = std::min(ESContrib, (int) TTI->getShuffleCost(
lib/Transforms/Vectorize/BBVectorize.cpp:              if (ShuffleVectorInst *SI = dyn_cast<ShuffleVectorInst>(*I)) {
lib/Transforms/Vectorize/BBVectorize.cpp:                // Shuffle can be folded if it has no other input
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib = (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib = std::min(ESContrib, (int) TTI->getShuffleCost(
lib/Transforms/Vectorize/BBVectorize.cpp:                // type, then they can be replaced with a shuffle
lib/Transforms/Vectorize/BBVectorize.cpp:                // If both are a shuffle with equal operand types and only two
lib/Transforms/Vectorize/BBVectorize.cpp:                // shuffle
lib/Transforms/Vectorize/BBVectorize.cpp:                ShuffleVectorInst *SIO1 = dyn_cast<ShuffleVectorInst>(O1),
lib/Transforms/Vectorize/BBVectorize.cpp:                                  *SIO2 = dyn_cast<ShuffleVectorInst>(O2);
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib = (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:                  ESContrib = std::min(ESContrib, (int) TTI->getShuffleCost(
lib/Transforms/Vectorize/BBVectorize.cpp:                // both need to be shuffled together.
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib += (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:                // both need to be shuffled together.
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib += (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:                ESContrib = (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:                  ESContrib += (int) getInstrCost(Instruction::ShuffleVector,
lib/Transforms/Vectorize/BBVectorize.cpp:  void BBVectorize::fillNewShuffleMask(LLVMContext& Context, Instruction *J,
lib/Transforms/Vectorize/BBVectorize.cpp:      int m = cast<ShuffleVectorInst>(J)->getMaskValue(v);
lib/Transforms/Vectorize/BBVectorize.cpp:  // Returns the value that is to be used as the vector-shuffle mask to the
lib/Transforms/Vectorize/BBVectorize.cpp:  Value *BBVectorize::getReplacementShuffleMask(LLVMContext& Context,
lib/Transforms/Vectorize/BBVectorize.cpp:    // This is the shuffle mask. We need to append the second
lib/Transforms/Vectorize/BBVectorize.cpp:    fillNewShuffleMask(Context, I, 0,        NumInElemJ, NumInElemI,
lib/Transforms/Vectorize/BBVectorize.cpp:    fillNewShuffleMask(Context, J, NumElemI, NumInElemI, NumInElemJ,
lib/Transforms/Vectorize/BBVectorize.cpp:    // exist). We might need a shuffle.
lib/Transforms/Vectorize/BBVectorize.cpp:    ShuffleVectorInst *LSV = dyn_cast<ShuffleVectorInst>(LOp);
lib/Transforms/Vectorize/BBVectorize.cpp:    ShuffleVectorInst *HSV = dyn_cast<ShuffleVectorInst>(HOp);
lib/Transforms/Vectorize/BBVectorize.cpp:    // FIXME: If we're fusing shuffle instructions, then we can't apply this
lib/Transforms/Vectorize/BBVectorize.cpp:    // optimization. The input vectors to the shuffle might be a different
lib/Transforms/Vectorize/BBVectorize.cpp:    // length from the shuffle outputs. Unfortunately, the replacement
lib/Transforms/Vectorize/BBVectorize.cpp:    // shuffle mask has already been formed, and the mask entries are sensitive
lib/Transforms/Vectorize/BBVectorize.cpp:    bool IsSizeChangeShuffle =
lib/Transforms/Vectorize/BBVectorize.cpp:      isa<ShuffleVectorInst>(L) &&
lib/Transforms/Vectorize/BBVectorize.cpp:    if ((LEE || LSV) && (HEE || HSV) && !IsSizeChangeShuffle) {
lib/Transforms/Vectorize/BBVectorize.cpp:          // A shuffle is needed.
lib/Transforms/Vectorize/BBVectorize.cpp:            new ShuffleVectorInst(I1, UndefValue::get(I1T),
lib/Transforms/Vectorize/BBVectorize.cpp:        // smaller one will need to grow before they can be shuffled together.
lib/Transforms/Vectorize/BBVectorize.cpp:            new ShuffleVectorInst(I1, UndefValue::get(I1T),
lib/Transforms/Vectorize/BBVectorize.cpp:            new ShuffleVectorInst(I2, UndefValue::get(I2T),
lib/Transforms/Vectorize/BBVectorize.cpp:        // Now that both I1 and I2 are the same length we can shuffle them
lib/Transforms/Vectorize/BBVectorize.cpp:          new ShuffleVectorInst(I1, I2, ConstantVector::get(Mask),
lib/Transforms/Vectorize/BBVectorize.cpp:        // The two vector inputs to the shuffle must be the same length,
lib/Transforms/Vectorize/BBVectorize.cpp:          NLOp = new ShuffleVectorInst(LOp, UndefValue::get(ArgTypeL),
lib/Transforms/Vectorize/BBVectorize.cpp:          NHOp = new ShuffleVectorInst(HOp, UndefValue::get(ArgTypeH),
lib/Transforms/Vectorize/BBVectorize.cpp:      Instruction *BV = new ShuffleVectorInst(LOp, HOp,
lib/Transforms/Vectorize/BBVectorize.cpp:      } else if (isa<ShuffleVectorInst>(I) && o == NumOperands-1) {
lib/Transforms/Vectorize/BBVectorize.cpp:        ReplacedOperands[o] = getReplacementShuffleMask(Context, I, J);
lib/Transforms/Vectorize/BBVectorize.cpp:  // original I and J instructions. These are generally vector shuffles
lib/Transforms/Vectorize/BBVectorize.cpp:        K1 = new ShuffleVectorInst(K, UndefValue::get(VType),
lib/Transforms/Vectorize/BBVectorize.cpp:        K2 = new ShuffleVectorInst(K, UndefValue::get(VType),
lib/Transforms/Vectorize/BBVectorize.cpp:        // flip it if that will yield fewer shuffles. We count the number
lib/Transforms/Vectorize/LoopVectorize.cpp:  /// instruction (shuffle) for loop invariant values and for the induction
lib/Transforms/Vectorize/LoopVectorize.cpp:  /// Generate a shuffle sequence that will reverse the vector Vec.
lib/Transforms/Vectorize/LoopVectorize.cpp:  SmallVector<Constant*, 8> ShuffleMask;
lib/Transforms/Vectorize/LoopVectorize.cpp:    ShuffleMask.push_back(Builder.getInt32(VF - i - 1));
lib/Transforms/Vectorize/LoopVectorize.cpp:  return Builder.CreateShuffleVector(Vec, UndefValue::get(Vec->getType()),
lib/Transforms/Vectorize/LoopVectorize.cpp:                                     ConstantVector::get(ShuffleMask),
lib/Transforms/Vectorize/LoopVectorize.cpp:           isa<ShuffleVectorInst>(I) || isa<GetElementPtrInst>(I);
lib/Transforms/Vectorize/LoopVectorize.cpp:      // VF is a power of 2 so we can emit the reduction using log2(VF) shuffles
lib/Transforms/Vectorize/LoopVectorize.cpp:      SmallVector<Constant*, 32> ShuffleMask(VF, 0);
lib/Transforms/Vectorize/LoopVectorize.cpp:          ShuffleMask[j] = Builder.getInt32(i/2 + j);
lib/Transforms/Vectorize/LoopVectorize.cpp:        std::fill(&ShuffleMask[i/2], ShuffleMask.end(),
lib/Transforms/Vectorize/LoopVectorize.cpp:        Builder.CreateShuffleVector(TmpVec,
lib/Transforms/Vectorize/LoopVectorize.cpp:                                    ConstantVector::get(ShuffleMask),
lib/Transforms/Vectorize/LoopVectorize.cpp:      Cost += TTI.getShuffleCost(TargetTransformInfo::SK_Reverse,
lib/Transforms/Vectorize/SLPVectorizer.cpp:      return TTI->getShuffleCost(TargetTransformInfo::SK_Broadcast, VecTy, 0);
lib/Transforms/Vectorize/SLPVectorizer.cpp:/// \brief Generate a shuffle mask to be used in a reduction tree.
lib/Transforms/Vectorize/SLPVectorizer.cpp:static Value *createRdxShuffleMask(unsigned VecLen, unsigned NumEltsToRdx,
lib/Transforms/Vectorize/SLPVectorizer.cpp:  SmallVector<Constant *, 32> ShuffleMask(
lib/Transforms/Vectorize/SLPVectorizer.cpp:      ShuffleMask[i] = Builder.getInt32(2 * i + !IsLeft);
lib/Transforms/Vectorize/SLPVectorizer.cpp:      ShuffleMask[i] = Builder.getInt32(NumEltsToRdx + i);
lib/Transforms/Vectorize/SLPVectorizer.cpp:  return ConstantVector::get(ShuffleMask);
lib/Transforms/Vectorize/SLPVectorizer.cpp:          createRdxShuffleMask(ReduxWidth, i, true, true, Builder);
lib/Transforms/Vectorize/SLPVectorizer.cpp:          createRdxShuffleMask(ReduxWidth, i, true, false, Builder);
lib/Transforms/Vectorize/SLPVectorizer.cpp:        Value *LeftShuf = Builder.CreateShuffleVector(
lib/Transforms/Vectorize/SLPVectorizer.cpp:        Value *RightShuf = Builder.CreateShuffleVector(
lib/Transforms/Vectorize/SLPVectorizer.cpp:          createRdxShuffleMask(ReduxWidth, i, false, false, Builder);
lib/Transforms/Vectorize/SLPVectorizer.cpp:        Value *Shuf = Builder.CreateShuffleVector(
Robs-MacBook-Pro:llvm cameron$ 
lib/Target/X86/Utils/X86ShuffleDecode.cpp://===-- X86ShuffleDecode.cpp - X86 shuffle decode logic -------------------===//
lib/Target/X86/Utils/X86ShuffleDecode.cpp:// Define several functions to decode x86 specific shuffle semantics into a
lib/Target/X86/Utils/X86ShuffleDecode.cpp:#include "X86ShuffleDecode.h"
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodeINSERTPSMask(unsigned Imm, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  ShuffleMask.push_back(0);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  ShuffleMask.push_back(1);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  ShuffleMask.push_back(2);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  ShuffleMask.push_back(3);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  ShuffleMask[CountD] = InVal;
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  if (ZMask & 1) ShuffleMask[0] = SM_SentinelZero;
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  if (ZMask & 2) ShuffleMask[1] = SM_SentinelZero;
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  if (ZMask & 4) ShuffleMask[2] = SM_SentinelZero;
lib/Target/X86/Utils/X86ShuffleDecode.cpp:  if (ZMask & 8) ShuffleMask[3] = SM_SentinelZero;
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodeMOVHLPSMask(unsigned NElts, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:    ShuffleMask.push_back(NElts+i);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:    ShuffleMask.push_back(i);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodeMOVLHPSMask(unsigned NElts, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:    ShuffleMask.push_back(i);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:    ShuffleMask.push_back(NElts+i);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:                       SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(Base + l);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:/// DecodePSHUFMask - This decodes the shuffle masks for pshufd, and vpermilp*.
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodePSHUFMask(MVT VT, unsigned Imm, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(NewImm % NumLaneElts + l);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:                       SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(l + i);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(l + 4 + (NewImm & 3));
lib/Target/X86/Utils/X86ShuffleDecode.cpp:                       SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(l + (NewImm & 3));
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(l + i);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:/// DecodeSHUFPMask - This decodes the shuffle masks for shufp*. VT indicates
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodeSHUFPMask(MVT VT, unsigned Imm, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:        ShuffleMask.push_back(NewImm % NumLaneElts + s + l);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:/// DecodeUNPCKHMask - This decodes the shuffle masks for unpckhps/unpckhpd
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodeUNPCKHMask(MVT VT, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(i);          // Reads from dest/src1
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(i+NumElts);  // Reads from src/src2
lib/Target/X86/Utils/X86ShuffleDecode.cpp:/// DecodeUNPCKLMask - This decodes the shuffle masks for unpcklps/unpcklpd
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodeUNPCKLMask(MVT VT, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(i);          // Reads from dest/src1
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(i+NumElts);  // Reads from src/src2
lib/Target/X86/Utils/X86ShuffleDecode.cpp:                          SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:    return; // Not a shuffle
lib/Target/X86/Utils/X86ShuffleDecode.cpp:      ShuffleMask.push_back(i);
lib/Target/X86/Utils/X86ShuffleDecode.cpp:/// DecodeVPERMMask - this decodes the shuffle masks for VPERMQ/VPERMPD.
lib/Target/X86/Utils/X86ShuffleDecode.cpp:void DecodeVPERMMask(unsigned Imm, SmallVectorImpl<int> &ShuffleMask) {
lib/Target/X86/Utils/X86ShuffleDecode.cpp:    ShuffleMask.push_back((Imm >> (2*i)) & 3);
lib/Target/X86/Utils/X86ShuffleDecode.h://===-- X86ShuffleDecode.h - X86 shuffle decode logic -----------*-C++-*---===//
lib/Target/X86/Utils/X86ShuffleDecode.h:// Define several functions to decode x86 specific shuffle semantics into a
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodeINSERTPSMask(unsigned Imm, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodeMOVHLPSMask(unsigned NElts, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodeMOVLHPSMask(unsigned NElts, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodePALIGNRMask(MVT VT, unsigned Imm, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodePSHUFMask(MVT VT, unsigned Imm, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodePSHUFHWMask(MVT VT, unsigned Imm, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodePSHUFLWMask(MVT, unsigned Imm, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:/// DecodeSHUFPMask - This decodes the shuffle masks for shufp*. VT indicates
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodeSHUFPMask(MVT VT, unsigned Imm, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:/// DecodeUNPCKHMask - This decodes the shuffle masks for unpckhps/unpckhpd
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodeUNPCKHMask(MVT VT, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:/// DecodeUNPCKLMask - This decodes the shuffle masks for unpcklps/unpcklpd
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodeUNPCKLMask(MVT VT, SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:                          SmallVectorImpl<int> &ShuffleMask);
lib/Target/X86/Utils/X86ShuffleDecode.h:/// DecodeVPERMMask - this decodes the shuffle masks for VPERMQ/VPERMPD.
lib/Target/X86/Utils/X86ShuffleDecode.h:void DecodeVPERMMask(unsigned Imm, SmallVectorImpl<int> &ShuffleMask);