秒杀项目
优极限【完整项目实战】半天带你用SpringBoot、Redis轻松实现Java高并发秒杀系统
技术栈
SpringBoot + MP
中间件:
RabbitMQ:异步、解耦系统中的一些模块、流量削峰作用
Redis:缓存
课程介绍
- 项目搭建
- 分布式Session: 秒杀-> 商城 -> 微服务 -> 分布式 -> 分布式共享Session
- 秒杀功能:增删改查
- 压力测试:超卖、并发量
- 页面优化
- 服务优化:异步、接口优化:Redis的预减库存、内存标记Redis、减少Redis的访问、分布式锁
- 接口安全:秒杀地址隐藏、黄牛脚本、验证码、接口限流
学习目标
安全优化
服务优化
- RabbitMQ消息队列:缓冲、异步下单
- 接口优化:从数据库到Redis,到网络通信、到内存标记
- 分布式锁:控制库存
页面优化
分布式会话
功能开发
系统压测
如何设计一个秒杀系统
稳、准、快:高可用、数据一致性、高性能
- 高性能
- 秒杀涉及大量并发读和写,动静分离方案、热点的发现和隔离、请求的削峰与分层过滤、服务端的极致优化
- 高可用
- 保证系统的高可用和准确性,还要设计一个PlanB来兜底
- 一致性
- 有限数量的商品在同一时刻被很多倍的请求同时减库存,减库存分为:“拍下减库存”、“付款减库存”以及预扣等,保证数据的准确性
应对高并发:缓存、异步、安全用户
解决:并发读、并发写
- 并发读
- 并发写
- 针对秒杀系统做保护
- 意料之外的情况设计兜底方案
项目搭建
配置文件:
hikari:
pool-name: DateHikariCP
minimum-idle: 5
idle-timeout: 1800000
maximum-pool-size: 10
max-lifetime: 1800000
connection-timeout: 30000
connection-test-query: SELECT 1
mybatis-plus:
mapper-locations: classpath* : /mapper/*Mapper.xml
type-aliases-package: com.xxXx.seckill.pojo
logging:
level:
com.XXXx.seckill.mapper: debug
分布式会话
登录功能
数据库
CREATE TABLE t_user(
`id` BIGINT(20) NOT NULL COMMENT '用户ID,手机号码',
`nickname` VARCHAR(255) NOT NULL,
`password` VARCHAR(32) DEFAULT NULL CONENT 'MD5(MD5(pass明文+固定salt)+salt)',
`salt` VARCHAR(10) DEFAULT NULL,
`head` VARCHAR(128) DEFAULT NULL COMMENT '头像',
`register_date` datetime DEFAULT NULL COMMENT'注册时间',
`last_login_date` datetime DEFAULT NULL COMMENT '最后一次登录时间',`login_count` int(11) DEFAULT '0' COMMENT '登录次数',
PRIMARY KEY(`id`)
)
两次MD5加密:保证安全
- 第一次:用户输入明文密码,传到后端,明文密码在网络中传输容易被截获
- 第二次:后端接到已完成第一次MD5加密的数据在存到数据库之前再进行一次MD5加密
MD5工具类
public class MD5Util {
public static string md5(string src){
return Digestutils.md5Hex(src);
}
private static final String salt="1a2b3c4d" ;
public static String inputPassToFromPass(String inputPass){
String str = salt.charAt(0)+salt.charAt(2)+inputPass+salt.charAt(5)+salt.charAt(4);
return md5(str);
}
public static String formPassToDBPass(String formPass,String salt){
String str = salt.charAt(0)+salt.charAt(2)+inputPass+salt.charAt(5)+salt.charAt(4);
return md5(str);
}
public static String inputPassToDBPass(string inputPass,string salt){
String fromPass = inputPassToFromPass(inputPass);
String dbPass = formPassToDBPass(fromPass,salt);
return dbPass;
}
}
参数校验
自定义注解参数校验
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
有了自定义注解要有自定义规则
@Target({ METHOD,FIELD,ANNOTATION_TYPE,CONSTRUCTOR,PARANETER,TYPE_USE})
@Retention(RUNTIME)
@Documented
@Constraint(validatedBy = {IsMobilevalidator.class})
public @interface IsHobile {
boolean required() default true;
String message() default "手机号码格式错误";
Class<?>[] groups() default { };
Class<? extends Payload>[] payload() default { };
}
自定义规则实现类,把自定义规则写进去
public class IsMobileValidator implements ConstraintValidator<IsNobile ,String>{
private boolean required = false;
@Override
public void initialize(IsMobile constraintAnnotation) {
required = constraintAnnotation.required();
}
@Override
public boolean isValid(String value,ConstraintValidatorContext context) {
if (required){
return ValidatorUtil.isMobile(value);
}else {
if (stringUtils.isEmpty(value)){
return true;
}else {
return ValidatorUtil.isHobile(value);
}
}
}
}
异常处理
分布式Session
CookieUtil
UUIDUtil
public class UUIDUtil {
public static String uuid() {
return UUID.randomuuID().toString().replace( target: "-",replacement: "");
}
}
生成Cookie
String ticket = UUIDUtil.uuid();
request.getSession().setAttribute(ticket , user);
CookieUtil.setCookie(request,response, "userTicket" ,ticket);
return RespBean.success();
分布式Session问题
刚开始我们在Tomcat1登录之后,用户信息放在Tomcat1的Session里。过了一会,请求又被Nginx分发到了Tomcat2上,这时Tomcat2 上 session里还没有用户信息,于是又要登录。
解决方案:
- Session复制
- 优点:无需修改代码,只修改Tomcat配置
- 缺点:Session同步传输占用内网带宽,多台Tomcat同步性能指数级下降,Session占用太多内存,无法有效水平扩展
- 前端存储
- 优点:不占用服务端内存
- 缺点:占用外网带宽,存在安全风险,数据大小受cookie限制
- Session粘滞
- 优点:无需修改代码,服务端可以水平扩展
- 缺点:增加新机器,会重新Hash,导致重新登录,应用重启需要重新登录
- 后端集中存储
- 优点:安全,容易水平扩展
- 缺点:增加复杂度,需要修改代码
- JWT方式,利用token
Redis存储用户信息
优化登录功能
通过MVC 即 ArgumentResolver 不用每次都判断用户信息,可以直接在Controller里获取用户信息
@Configuration
@EnableWebMvc
public class WebConfig implements WebMvcConfigurer {
@Autowired
private UserArgumentResolver userArgumentResolver;
@Override
public void addArgumentResolvers(List<HandlerHethodArgumentResolver> resolvers){
resolvers.add(userArgumentResolver);
}
}
@Component
public class UserArgumentResolver implements HandlerMethodArgumentResolver {
@Autowired
private IUserservice userService;
@Override
public boolean supportsParameter(MethodParameter parameter) {
Class<?> clazz = parameter.getParameterType();
return clazz== User.class;
}
@Override
public Object resolveArgument(NethodParameter parameter,ModelAndViewContainer mavContainerNativeWebRequest webRequest,WebDataBinderFactory binderFactory) throws Exception {
HttpServletRequest request = webRequest.getNativeRequest(HttpServletRequest.class);
HttpServletResponse response = webRequest.getNativeResponse(HttpServletResponse.class);
String ticket = CookieUtil.getCookieValue(request,cookieName: "userTicket" );
if (stringutils.isEmpty(ticket)) {
return null;
}
return userService.getUserByCookie(ticket,request,response);
}
}
秒杀功能
商品表、秒杀表、秒杀订单表、订单表
数据库
CREATE TABLE `t_goods`(
`id` BIGINT(20) NOT NULL AUTO_INCREMENT COMMENT '商品ID',
`goods_name` VARCHAR(16) DEFAULT NULL COMMENT '商品名称',
`goods_title` VARCHAR(64) DEFAULT NULL COMMENT '商品标题',
`goods_img` VARCHAR(64) DEFAULT NULL COMMENT '商品图片',
`goods_detail` LONGTEXT COMMENT '商品详情',
`goods_price` DECIMAL(10,2) DEFAULT '0.00' COMMENT '商品价格',
`goods_stock` INT(11) DEFAULT '0' COMMENT '商品库存,-1表示没有限制',
PRIMARY KEY(`id`)
)ENGINE = INNODB AUTO_INCREMENT=3 DEFAULT CHARSET= utf8mb4;
CREATE TABLE `t_order`(
`id` BIGINT(20) NOT NULL AUTO_INCREMENT COMMENT '订单ID',
`user_id` BIGINT(20) DEFAULT NOT NULL COMMENT '用户ID',
`goods_id` BIGINT(20) DEFAULT NOT NULL COMMENT '商品ID',
`delivery_addr_id` BIGINT(20) DEFAULT NOT NULL COMMENT '收货地址ID',
`goods_name` VARCHAR(16) DEFAULT NULL COMMENT'冗余过来的商品名称',
`goods_count` INT(11)DEFAULT '0'COMMENT '商品数量',
`goods_price` DECIMAL(10,2) DEFAULT '0.00' COMMENT '商品单价',
`order_channel` TINYINT(4) DEFAULT '0' COMMENT '1pc, 2android,3ios',
`status` TINYINT(4) DEFAULT '0' COMMENT '订单状态,0新建未支付,1已支付,2已发货,3已收货,4己退款,5已完成',
`create_date` datetime DEFAULT NULL COMMENT '订单的创建时间',
`pay_date` datetime DEFAULT NULL COMMENT '支付时间'·
PRIMARY KEY(`id`)
)ENGINE = INNODB AUTO_INCREMENT=3 DEFAULT CHARSET= utf8mb4;
CREATE TABLE `t_seckill_goods`(
`id` BIGINT(20) NOT NULL AUTO_INCREMENT COMMENT '秒杀商品ID',
`goods_id` BIGINT(20) DEFAULT NULL COMMENT '商品ID',
`seckill_price` DECIMAL(10,2) DEFAULT '0.00' COMMENT '秒杀价',
`stock_count INT(10) DEFAULT NULL COMMENT '库存数量',
`start_date` datetime DEFAULT NULL COMMENT '秒杀开始时间',
`end_date` datetime DEFAULT NULL COMMENT '秒杀结束时间',
PRIMARY KEY(`id`)
)ENGINE = INNODB AUTO_INCREMENT=3 DEFAULT CHARSET= utf8mb4;
#秒杀订单表
CREATE TABLE `t_seckill_order`(
`id` BIGINT(20) NOT NULL AUTO_INCREMENT COMMENT '秒杀订单ID',
`user_id` BIGINT(20) DEFAULT NOT NULL COMMENT '用户ID',
`order_id` BIGINT(20) DEFAULT NOT NULL COMMENT '订单ID',
`goods_id` BIGINT(20) DEFAULT NOT NULL COMMENT '商品ID',
PRIMARY KEY(`id`)
)ENGINE = INNODB AUTO_INCREMENT=3 DEFAULT CHARSET= utf8mb4;
实现商品列表页
商品名称、商品图片、商品原价、秒杀价、库存数量、详情
SELECT
g.id,
g.goods_name,g.goods_title,g.goods_img,
g.goods_detail,g.goods_price,g.goods_stock,
sg.seckill_price,sg.stock_count,sg.start_date,sg.end_date
FROM
t_goods g
LEFT J0IN t_seckill_goods AS sg ON g.id = sg.goods_id
实现商品详情页
商品名称、商品图片、秒杀开始时间、商品原价、秒杀价、库存数量
SELECT
g.id,
g.goods_name,g.goods_title,g.goods_img,g.goods_detail,g.goods_price,g.goods_stock,
sg.seckill_price,sg.stock_count,sg.start_date,sg.end_date
FROM
t_goods g
LEFT J0IN t_seckill_goods As sg oN g.id = sg.goods_id
WHERE
g.id =
秒杀倒计时
时间格式化:在实体类中的时间字段上添加@JsonFormat注解
@RequestMapping("/toDetail/{goodsId}")
public String toDetail(Hodel model,User user,@PathVariable Long goodsId){
model.addAttribute( "user" , user);
GoodsVo goodsVo = goodsService.findGoodsVoBy6oodsId(goodsId);
Date startDate = goodsVo.getstartDate();
Date endDate = goodsVo. getEndDate();
Date nowDate = new Date();
int secKillStatus = 0;
int remainSeconds = 0;
if (nowDate.before(startDate)){
remainSeconds = ((int) ((startDate.getTime() - nowDate.getTime())/ 1000));
}else if (nowDate.after(endDate)){
secKillStatus = 2;
remainSeconds = -1;
}else {
secKillstatus = 1;
remainSeconds = 0;
}
model.addAttribute( "remainSeconds" , remainSeconds);
model.addAttribute( "secKillstatus" ,seckillstatus);
model.addAttribute( "goods" , goodsVo);
return "goodsDetail";
}
前端
<tr>
<td>秒杀开始时间</td>
<td th:text="${#dates.format(goods.startDate, ' vvvy-MN-dd HH:mm:ss')}"></td>
<td id="seckillTip">
<input type="hidden" id="remainseconds" th:value="$iremainSeconds}">
<span th:if="${seckillStatus eq 0}">秒杀倒计时:<span id="countDown" th:text="${remainSeconds}"></span>秒
</span>
<span th:if="${secKillStatus eq 1}">秒杀进行中</span>
<span th: if="$isecKillStatus eq 2}">秒杀已结束</span>
</td>
</tr>
<script>
$ (function (){
countDown();
});
function countDown(){
var remainSeconds = $("#remainSeconds" ).val();
var timeout;
if (remainseconds > 0){
timeout = setTimeout(function (){
$("#countDown" ).text(remainSeconds - 1);
$("#remainSeconds" ).val(remainSeconds - 1);
countDown();
},1000) ;
}else if (remainSeconds == 0){
if (timeout){
clearTimeout(timeout);
}
$("#seckillTip").html("秒杀进行中")
}else {
$("#seckil1Tip").html("秒杀已经结束");
}
};
</script>
秒杀按钮
<td>
<form id="secKillForm" method="post" action="/seckill/doSeckill">
<input type="hidden" name="goodsId" th: value="${goods.id}">
<button class="btn btn-primary btn-block" type="submit" id="buyButton">立即秒杀</button>
</form>
</td>
<script>
$ (function (){
countDown();
});
function countDown(){
var remainSeconds = $("#remainSeconds" ).val();
var timeout;
if (remainseconds > 0){
$("#buyButton" ).attr("disabled",true);
timeout = setTimeout(function (){
$("#countDown" ).text(remainSeconds - 1);
$("#remainSeconds" ).val(remainSeconds - 1);
countDown();
},1000) ;
}else if (remainSeconds == 0){
$("#buyButton" ).attr("disabled",false);
if (timeout){
clearTimeout(timeout);
}
$("#seckillTip").html("秒杀进行中")
}else {
$("#buyButton" ).attr("disabled",true);
$("#seckil1Tip").html("秒杀已经结束");
}
};
</script>
秒杀功能实现
库存够不够、用户不能重复秒杀
@RequestMapping("/doSecKill")
public String doSeckill(Model model,User user,Long goodsId) {
if (user == null) {
return "login" ;
}
model.addAttribute("user", user);
GoodsVo goods = goodsservice.findGoodsVoByGoodsId(goodsId);
if (goods.getstockCount() < 1) {
model.addAttribute(attributeName: "errmsg",RespBeanEnum.EINIPTY_STOcK.getNessage());
return "secKillFail";
}
Seckill0rder seckill0rder = seckillorderService.getone(new QueryWrapper<Seckill0rder>().eq( "user_id",user.getId
()).eq("goods_id",goodsId));
if (seckill0rder != null) {
model.addAttribute("errmsg",RespBeanEnum.REPEATE_ERROR.getMessage())
return "secKillFail";
}
Order order = orderservice.seckill(user, goods);
model.addAttribute("order",order);
model.addAttribute("goods",goods);
return "orderDetail" ;
}
@Override
public Order seckill(User user, GoodsVo goods) {
SeckillGoods seckillGoods = seckillGoodsService.getOne(new QueryWIrapper<SeckillGoods>().eq("goods_id",goods.getId()));
seckillGoods.setStockCount(seckillGoods.getstockCount()-1);
seckillGoodsService. updateById(seckillGoods) ;
Order order = new Order();
order.setUserId(user.getId());
order.setGoodsId(goods.getId();
order.setDeliveryAddrId(0L);
order.setGoodsName(goods.getGoodsName());
order.setGoodsCount(1);
order.setGoodsPrice(seckillGoods.getseckillPrice());
order.set0rderChannel(1);
order.setstatus(0);
order.setCreateDate(new Date());
orderMapper.insert(order);
SeckillOrderr seckillOrder = new SeckillOrder();
seckillOrder.setuserId(user.getId());
seckillOrder.setOrderId(order.getId());
seckillOrder.setGoodsId(goods.getId());
seckillOrderService.save(seckil1Order);
return order;
}
系统压测
QPS:每秒查询率,一台服务器每秒查询次数,特定的查询服务器在规定时间内所处理流量多少的标准
TPS:事务/秒,软件测试结果的测量单位,一个客户机向服务器发送请求,服务器做出响应的过程
JMeter的使用
测试计划:
- 添加 -> 线程 -> 线程组
- 线程属性:线程数、Ramp-Up时间(几秒钟之内启动线程数)、循环次数
- 添加 -> 配置元件 -> HTTP请求默认值
- Web服务器:协议:HTTP、IP地址:localhost、端口:8080
- 添加 -> 取样器 -> HTTP请求
- 添加 -> 监听器 -> 查看结果数、聚合报告、用表格查看结果
在Linux里运行JMeter
- 在Linux里安装MySQL,或将项目地址改成本机地址
- 把项目打包成jar扔到服务器上
- 把JMeter扔到服务器上去,解压之后直接使用
- 通过命令把运行脚本(Windows版本的线程组测试脚本创建好)扔进去
- 将生成的报告文件扔出来放到Windows版本里查看
配置同一用户测试
添加 -> 取样器 -> HTTP请求
配置不同用户测试
添加 -> 配置元件 -> CSV Data Set Config
添加 -> 配置元件 -> HTTP Cookie管理器
添加 -> 取样器 -> HTTP请求
此处发现问题:
- Linux和Windows下的优化前QPS差距过大
- 库存出现负数,出现超卖问题
页面优化
第一个优化:添加缓存
QPS最大的瓶颈在于数据库的操作,可以将数据库的操作提取出来放入缓存,(前提是该缓存频繁被读取且变更比较少)
@RequestNapping(value = "/toList",produces = "text/html;charset=utf-8")
@ResponseBody
public String toList(Model model,User user,HttpServletRequest request,HttpServletResponse response) {
ValueOperations valueOperations = redisTemplate.opsForValue();
String html = (String) value0perations.get("goodsList");
if (!stringutils.isEmpty(html)) {
return html;
}
model.addAttribute("user", user);
model.addAttribute("goodsList",goodsService.findGoodsVo());
WebContext context = new WebContext(request,response,request.getServletContext(),request.getLocale(),
model.asMap());
html = thymeleafViewResolver.getTemplateEngine().process("goodsList",context);
if(!StringUtils.isEmpty(html)){
valueoperations.set("goodsList" , html,60,TimeUnit.SECONDS);
}
return html;
}
@RequestMapping(value = "/toDetail/{goodsId}" , produces = "text/html;charset=utf-8")
@ResponseBody
public String toDetail(Nodel model,User user,@PathVariable Long goodsId,HttpServletRequest request, HttpServletResponse response) {
ValueOperations valueOperations = redisTemplate.opsForValue();
String html = (String) value0perations.get("goodsDetail:" + goodsId);
if(!StringUtils.isEmpty(html)){
return html;
}
model.addAttribute( "user" , user);
GoodsVo goodsVo = goodsService.findGoodsVoBy6oodsId(goodsId);
Date startDate = goodsVo.getstartDate();
Date endDate = goodsVo. getEndDate();
Date nowDate = new Date();
int secKillStatus = 0;
int remainSeconds = 0;
if (nowDate.before(startDate)){
remainSeconds = ((int) ((startDate.getTime() - nowDate.getTime())/ 1000));
}else if (nowDate.after(endDate)){
secKillStatus = 2;
remainSeconds = -1;
}else {
secKillstatus = 1;
remainSeconds = 0;
}
model.addAttribute( "remainSeconds" , remainSeconds);
model.addAttribute( "secKillstatus" ,seckillstatus);
model.addAttribute( "goods" , goodsVo);
WebContext context = new WebContext(request,response,request.getservletContext(), request.ypetiocale(),
model.asMap());
thymeleafViewResolver.getTemplateEnaine( ).process("goodsDetail",context);
if (!StringUtils.isEmpty(html)) {
valueOperations.set("goodsDetail:" + goodsId,html,60,TimeUnit.SECONDS) ;
}
return html;
}
@Override
public RespBean updatePassword(String userTicke,String password,HttpServletRequest request,
HttpservletResponse response) {
User user = getUserByCookie(userTicket,request,response);
if (user == null) {
throw new GlobalException(RespBeanEnum.MOBILE_NOT_EXIST);
}
user.setPassword(MD5Util.inputPassToDBPass(password,user.getslat()));
int result = userMapper.updateById(user);
if (1 == result) {
redisTemplate.delete("user: " + userTicket);
return RespBean.success();
}
return RespBean.error(RespBeanEnum.PASSWORD_UPDATE_FAIL);
}
第二个优化:页面静态化
做一个异步处理,渲染和请求分开做,然后拿到结果后再套入进去
页面跳转到公共的返回对象,进行返回,通过静态页面跳转,并通过ajax获取静态数据,调接口获取数据,手动渲染
后端
@RequestMapping(value = "/toDetail/{goodsId}")
@ResponseBody
public RespBean toDetail(User user,@PathVariable Long goodsId) {
GoodsVo goodsVo = goodsService.findGoodsVoBy6oodsId(goodsId);
Date startDate = goodsVo.getstartDate();
Date endDate = goodsVo. getEndDate();
Date nowDate = new Date();
int secKillStatus = 0;
int remainSeconds = 0;
if (nowDate.before(startDate)){
remainSeconds = ((int) ((startDate.getTime() - nowDate.getTime())/ 1000));
}else if (nowDate.after(endDate)){
secKillStatus = 2;
remainSeconds = -1;
}else {
secKillstatus = 1;
remainSeconds = 0;
}
DetailVo detailVo = new DetailVo();
detailVo.setUser(user);
detailVo.setGoodsVo(goodsVo);
detailVo.setSecKillstatus(seckillstatus);
detailVo.setRemainSeconds(remainSeconds);
return RespBean.success(detailVo);
}
前端
<script>
$ (function (){
getDetails();
});
function getDetails(){
var goodsId = g_getQueryString( "goodsId");
$.ajax({
url: '/goods/detail/ '+goodsId,
type: 'GET',
success: function (data){
if (data.code==200){
render(data.obj);
}else {
layer.msg("客户端请求出错");
}
}
error: function (){
layer.msg("客户端请求出错");
}
});
}
function render(detail) {
var user = detail.user;
var goods = detail.goodsVo;
var remainSeconds = detail.remainSeconds;
if (user) {
$("#userTip" ).hide();
}
$("#goodsName").text(goods.goodsName);
$("#goodsImg").attr("src", goods.goodsImg);
$(" #startTime").text(new Date(goods.startDate).format("yyyy-MM-dd HH:mm:ss"));
$("#remainseconds").val(remainSeconds);
$("#goodsId").val(goods.id);
$("#goodsPrice").text(goods.goodsPrice);
$("#seckillPrice").text(goods.seckillPrice);
$("#stockCount").text(goods.stockCount);
countDown();
}
function countDown(){
var remainSeconds = $("#remainSeconds").val();
var timeout;
if (remainseconds > 0){
$("#buyButton").attr("disabled",true);
$("#seckillTip").html("秒杀倒计时" + remainSeconds + "秒");
timeout = setTimeout(function (){
$("#remainSeconds" ).val(remainSeconds - 1);
countDown();
},1000) ;
}else if (remainSeconds == 0){
$("#buyButton" ).attr("disabled",false);
if (timeout){
clearTimeout(timeout);
}
$("#seckillTip").html("秒杀进行中")
}else {
$("#buyButton" ).attr("disabled",true);
$("#seckil1Tip").html("秒杀已经结束");
}
};
</script>
后端
@PostMapping("/doSecKill")
@ResponseBody
public RespBean doSeckill(User user,Long goodsId) {
if (user == null) {
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
GoodsVo goods = goodsservice.findGoodsVoByGoodsId(goodsId);
if (goods.getstockCount() < 1) {
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
SeckillOrder seckillOrder = seckillOrderService.getOne(new QueryWrapper<SeckillOrder>().eq("user_id",user.getId()).eq("goods_id",goodsId));
if (seckilOrder != null) {
return RespBean.error(RespBeanEnum.REPEATE_ERROR);
}
Order order = orderservice.seckill(user, goods);
return RespBean.success(order) ;
}
前端
<script>
function doSeckill() {
$.ajax({
url: '/seckill/doSeckill',
type: 'POST',
data: {
goodsId: $("#goodsId").val()
},
success: function (data){
if (data.code == 200) {
window.location.href = "/orderDetail.htm?orderId=" + data.obj.id;
}else {
layer.msg("客户端请求错误");
}
},
error: function () {
layer.msg("客户端请求错误");
}
})
}
</script>
后端
@Override
public OrderDetailVo detail(Long orderId) {
if (order1d == null) {
throw new GlobalException(RespBeanEnum.ORDER_NOT_EXIST);
}
Order order = orderMapper.selectById(orderId);
GoodsVo goodsVo = goodsService.findGoodsVoByGoodsId(order.getGoodsId());
OrderDetailVo detail = new OrderDetailVo();
detail.setorder(order);
detail.setGoodsVo(goodsVo);
return detail;
}
前端
<script>
$(function () {
getOrderDetail();
});
function getOrderDetail() {
var orderId = g_getQueryString("orderId");
$.ajax({
url: '/order?detail',
type: 'GET',
data: {
orderId: orderId
},
success: function (data){
if (data.code == 200) {
render(data.obj);
}else {
layer.msg("客户端请求错误");
}
},
error: function () {
layer.msg("客户端请求错误");
}
})
}
function render(detail){
var goods = detail.goodsVo;
var order = detail.order;
$("#goodsName").text(goods.goodsName);
$("#goodsImg").attr("src",goods.goodsImg);
$("#goodsPrice").text(order.goodsPrice);
$("#createDate").text(new Date(order.createDate).format("yyyy-MN-dd HH:mm:ss"));
var status = order.status;
var statusText = "";
switch (status){
case 0:
statusText = "未支付";
break;
case 1:
statusText = "待发货";
break;
case 2:
statusText = "已发货";
break;
case 3:
statusText = "己收货";
break;
case 4:
statusText = "己退款";
break
case 5:
statusText = "已完成";
break;
}
$ ("#status").text(statusText);
}
</script>
解决库存超卖
减库存 -> 生成订单 -> 生成秒杀订单
而解决库存超卖需要做一些判断,判断商品库存是否大于0,判断时间节点是当你进行更新操作时,即更新操作时先判断库存
- 扣库存用sql语句处理,同时判断库存大于0
- 采用 用户id+商品id的唯一索引,解决同一个用户秒杀多个商品问题,虽然性能降低但是解决超卖问题
- 从Redis中判断是否重复抢购
@Transactional
@Override
public Order seckill(User user, GoodsVo goods) {
SeckillGoods seckillGoods = seckillGoodsService.getOne(new QueryWIrapper<SeckillGoods>().eq("goods_id",goods.getId()));
seckillGoods.setStockCount(seckillGoods.getstockCount() - 1);
boolean seckillGoodsResult = seckillGoodsService.update(new UpdateWrapper<SeckillGoods>().setSql( "stock_count = stock_count -1").eq("goods_id" , goods.getId())).gt("stock_count" , 0));
if(!seckillGoodsResult){
return null;
}
Order order = new Order();
order.setUserId(user.getId());
order.setGoodsId(goods.getId();
order.setDeliveryAddrId(0L);
order.setGoodsName(goods.getGoodsName());
order.setGoodsCount(1);
order.setGoodsPrice(seckillGoods.getseckillPrice());
order.set0rderChannel(1);
order.setstatus(0);
order.setCreateDate(new Date());
orderMapper.insert(order);
SeckillOrderr seckillOrder = new SeckillOrder();
seckillOrder.setuserId(user.getId());
seckillOrder.setOrderId(order. getId());
seckillOrder.setGoodsId(goods.getId());
seckillOrderService.save(seckil1Order);
redisTemplate.opsForValue().set("order:" + user.getId() + ":"+ goods.getId(), seckillOrder);
return order;
}
@PostMapping("/doSecKill")
@ResponseBody
public RespBean doSeckill(User user,Long goodsId) {
if (user == null) {
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
GoodsVo goods = goodsservice.findGoodsVoByGoodsId(goodsId);
if (goods.getstockCount() < 1) {
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
SeckillOrder seckillOrder =
(SeckillOrder) redisTemplate.opsForValue(). get("order:" + user.getId() + ":" + goodsId);
if (seckilOrder != null) {
return RespBean.error(RespBeanEnum.REPEATE_ERROR);
}
Order order = orderservice.seckill(user, goods);
return RespBean.success(order) ;
}
以上可发现优化后的QPS提升并不大,因为库存卖完后在判断同一个用户重复下单时放到了Redis,速度更快
还可继续优化的点
第三个优化:静态资源优化(略)
第四个优化:CDN优化(略)
服务优化
- 减库存:通过Redis预减库存,减少对数据库的访问,而Redis放在单独的服务器上,还需频繁和Redis进行网络通信,即再次进行优化,通过内存标记去减少对Redis的访问
- 下单:请求用到队列,先进入队列里进行缓冲,进行异步下单
Redis预减库存:在系统初始化时将商品数量加载到Redis中,当真正收到请求时通过Redis预减库存,库存不足则直接返回秒杀失败,如果库存充足则先将请求加入RabbitMQ消息队列,并且立即返回客户端正在排队中,请求入队之后,进行异步操作,异步生成订单,真正减少数据库库存,出单成功后在客户端做个轮询查询是否真正出了订单,出了订单即为秒杀成功,否则秒杀失败
增强数据库性能:将一个数据库,做集群,或者阿里巴巴的中间件MyCat对数据库进行分库分表,增强数据库性能
RabbitMQ
默认端口:15672;默认用户名密码:guest
SpringBoot整合RabbitMQ
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqps</artifactId>
</dependency>
rabbitmq:
host: 192.168.1.128
username: guest
password: guest
virtual-host: /
port: 5672
listener:
simple:
concurrency: 10
max-concurrency: 10
prefetch: 1
auto-startup: true
default-requeue-rejected: true
template:
retry:
enabled: true
initial-interval: 1000ms
max-attempts: 3
max-interval: 1000@ms
multiplier: 1
@Configuration
public class RabbitMQConfig {
@Bean
public Queue queue(){
return new Queue("queue",true);
}
}
@Service
@Slf4j
public class MQSender {
@Autowired
private RabbitTemplate rabbitTemplate;
public void send(Object msg) {
log.info("发送消息:" +msg);
rabbitTemplate.convertAndSend( "queue", msg);
}
}
@Service
@Slf4j
public class MQReceiver {
@RabbitListener(queues = "queue")
public void receive(object msg) {
log.info("接收消息:" +msg );
}
}
@Autowired
private MQSender mqSender;
@RequestMapping( "/mq")
@ResponseBody
public void mq(){
mqSender.send("Hello");
}
RabbitMQ交换机模式
交换机:一边接收来自生产者的消息,一边将消息推送到队列,交换机必须确切的知道如何处理接收到的消息,他的规则由交换机类型定义(direct、topic 、headers、fanout)
- Fanout模式(广播模式、发布订阅模式)
- 消息不仅仅被一个队列接收,而是能够被多个队列接收,多个队列接收的是同一个生产者发送的同一条消息
- 广播模式并不会处理路由键
@Configuration
public class RabbitMQConfig {
private static final String QUEUE01 = "queue_fanout01";
private static final String QUEUE02 = "queue_fanout02";
private static final String EXCHANGE = "fanoutExchange";
@Bean
public Queue queue(){
return new Queue( name: "queue", durable: true);
}
@Bean
public Queue queue01(){
return new Queue(QUEUE01);
}
@Bean
public Queue queue02(){
return new Queue(QUEUE02);
}
@Bean
public FanoutExchange fanoutExchange(){
return new FanoutExchange(EXCHANGE);
}
@Bean
public Binding binding01(){
return BindingBuilder.bind(queue01()).to(fanoutExchange());
}
@Bean
public Binding binding02(){
return BindingBuilder.bind(queue02()).to(fanoutExchange());
}
}
@Service
@Slf4j
public class MQSender {
@Autowired
private RabbitTemplate rabbitTemplate;
public void send(Object msg) {
log.info("发送消息:" +msg);
rabbitTemplate.convertAndSend("fanoutExchang","", msg);
}
}
@Service
@Slf4j
public class MQReceiver {
@RabbitListener(queues = "queue")
public void receive(object msg) {
log.info("接收消息:" +msg );
}
@RabbitListener(queves = "queue_fanout01")
public void receive01(Object msg) {
log.info("QUEUE01接收消息:" +msg);
}
@RabbitListener(queues = "queue_fanout02")
public void receive02(Object msg) {
log.info("QUEUEO2接收消息:" + msg);
}
}
- Direct模式(路由模式)
- 消息去到队列,绑定一个key,明确匹配了路由key
- 所有发送到Direct的j消息都会被转发到路由key中指定的一个Queue
- Direct可以使用RabbitMQ自带的交换机
@Configuration
public class RabbitMQConfig {
private static final String QUEUE01 = "queue_fanout01";
private static final String QUEUE02 = "queue_fanout02";
private static final String EXCHANGE = "directExchange";
private static final String ROUTINGKEY01 = "queue.red";
private static final String ROUTINGKEY02 = "queue.green";
@Bean
public Queue queue01(){
return new Queue(QUEUE01);
}
@Bean
public Queue queue02(){
return new Queue(QUEUE02);
}
@Bean
public DirectExchange directExchange(){
return new DirectExchange(EXCHANGE);
}
@Bean
public Binding binding01(){
return BindingBuilder.bind(queue01()).to(directExchange()).with(ROUTINGKEY01);
}
@Bean
public Binding binding02(){
return BindingBuilder.bind(queue02()).to(directExchange()).with(ROUTINGKEY02);
}
}
@Service
@Slf4j
public class MQSender {
@Autowired
private RabbitTemplate rabbitTemplate;
public void send01(Object msg) {
log.info("发送red消息:" +msg);
rabbitTemplate.convertAndSend("directExchange","queue.red", "msg");
}
public void send02(Object msg) {
log.info("发送green消息:" +msg);
rabbitTemplate.convertAndSend("directExchange","queue.green", "msg");
}
}
@Service
@Slf4j
public class MQReceiver {
@RabbitListener(queues = "queue")
public void receive(object msg) {
log.info("接收消息:" +msg );
}
RabbitListener(queues = "queue_direct01")
public void receive01(Object msg) {
log.info("QUEUE01接收消息:" + msg);
}
@RabbitListener(queues = "queue_direct02")
public void receive02(Object msg) {
log.info("QUEUE02接收消息:" +msg);
}
}
- Topic模式(主题模式) (常用)
- 为方便管理路由key引入通配符(#(匹配零个或多个)、*(匹配明确的一个))
@Configuration
public class RabbitMQConfig {
private static final String QUEUE01 = "queue_fanout01";
private static final String QUEUE02 = "queue_fanout02";
private static final String EXCHANGE = "topicExchange";
private static final String ROUTINGKEY01 = "#.queue.#" ;
private static final String ROUTINGKEYO2 = "*.queue.#";
@Bean
public Queue queue01(){
return new Queue(QUEUE01);
}
@Bean
public Queue queue02(){
return new Queue(QUEUE02);
}
@Bean
public TopicExchange topicExchange() {
return new TopicExchange(EXCHANGE);
}
@Bean
public Binding binding01() {
return BindingBuilder.bind(queue01()).to(topicExchange()).with(ROUTINGKEY01);
}
@Bean
public Binding binding02(){
return BindingBuilder.bind(queue02()).to(topicExchange()).with(ROUTINGKEY02);
}
}
@Service
@Slf4j
public class MQSender {
@Autowired
private RabbitTemplate rabbitTemplate;
public void send01(Object msg) {
log.info("发送消息(QUEUEO1接收):"+msg);
rabbitTemplate.convertAndSend("topicExchange","queue.red.message" ,msg)
}
public void send02(Object msg) {
log.info("发送消息(被两个queue接收):" + msg);
rabbitTemplate.convertAndSend("topicExchange","message.queue.green.abc", msg);
}
}
@Service
@Slf4j
public class MQReceiver {
@RabbitListener(queues = "queue")
public void receive(object msg) {
log.info("接收消息:" +msg );
}
@RabbitListener(queues = "queue_topic01")
private void receive01(Object msg) {
log.info("QUEUE01接收消恩:" + msg);
}
@RabbitListener(queues = "queue_topic02")
private void receive02(Object msg) {
log.info("QUEUE02接收消息:" + msg) ;
}
}
@Configuration
public class RabbitMQConfig {
private static final String QUEUE01 = "queue_fanout01";
private static final String QUEUE02 = "queue_fanout02";
private static final String EXCHANGE = "headerExchange";
@Bean
public Queue queue01(){
return new Queue(QUEUE01);
}
@Bean
public Queue queue02(){
return new Queue(QUEUE02);
}
@Bean
public HeadersExchange headersExchange(){
return new HeadersExchange(EXCHANGE);
}
@Bean
public Binding binding01(){
Map<String, Object> map = new HashMap<>();
map.put("color","red");
map.put("speed","low");
return BindingBuilderTbind(queue01()).to(headersExchange()).whereAny(map).match();
}
@Bean
public Binding binding02(){
Map<String, Object> map = new HashMap<>();
map.put("color","red");
map.put("speed","fast");
return BindingBuilderTbind(queue02()).to(headersExchange()).whereAll(map).match();
}
}
@Service
@Slf4j
public class MQSender {
@Autowired
private RabbitTemplate rabbitTemplate;
public void send01(Object msg) {
log.info("发送消息(被两个queue接收):" +msg);
MessageProperties properties = new MessageProperties();
properties.setHeader("color" , "red" );
properties.setHeader("speed" , "fast");
Message message = new Message(msg.getBytes() , properties);
rabbitTemplate.convertAndSend("headersExchange", "",message);
}
public void send02(Object msg){
log.info("发行消息(被QUEUE01接收):"+msg);
MessageProperties properties = new MessageProperties();
properties.setHeader("color" , "red");
properties.setHeader("speed" , "normal");
Message message = new Message(msg.getBytes() , properties);
rabbitTemplate.convertAndSend("headersExchange", "",message);
}
}
@Service
@Slf4j
public class MQReceiver {
@RabbitListener(queues = "queue")
public void receive(object msg) {
log.info("接收消息:" +msg );
}
@RabbitListener(queues = "queue_header01")
public void receive01(Message message) {
log.info("QUEUE01接收Message对象:" + message);
log.info("QUEUEO接收消息:" + new String(message.getBody()));
}
@RabbitListener(queues = "queue_header02")
public void receive02(Hessage message) {
log.info( "QUEUE02接收Message对象:" + message);
log.info("QUEUE02接收消息: " + new String(message.getBody()));
}
}
Redis预减库存
@Override
public void afterPropertiesset() throws Exception {
List<GoodsVo> list = goodsservice.findGoodsVo();
if (Collectionutils.isEmpty(list)) {
return;
}
list.forEach(goodsVo -> {
redisTemplate.opsForValue().set("seckillGoods:" + goodsVo.getId(),goodsVo.getStockCount());
});
}
@RequestMapping(value = "/doSeckill",method = RequestHethod.POST)
@ResponseBody
public RespBean doSeckill(User user,Long goodsId){
if (user == null) {
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
ValueOperations valueOperations = redisTemplate.opsForValue();
SeckillOrder seckillOrder =
(SeckillOrder) redisTemplate.opsForValue().get("order:" + user.getId() + ":" + goodsId);
if (seckillOrder != null) {
return RespBean.error(RespBeanEnum.REPEATE_ERROR);
}
Long stock = valueOperations.decrement( "seckillGoods:" + goodsId);
if (stock < 0) {
valueOperations.increment("seckillGoods: " + goodsId);
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
Order order = orderService.seckill(user,goods) ;
return RespBean.success(order);
}
RabbitMQ秒杀操作
封装了一个消息对象,通过RabbitMQ发送消息对象,在监听者里做了之前在Controller里做的事(判断库存、判断是否重复抢购、下单操作),使用RabbitMQ变成了异步操作,可以在Controller中快速返回,进行一个流量削峰的作用
@Override
public void afterPropertiesset() throws Exception {
List<GoodsVo> list = goodsservice.findGoodsVo();
if (Collectionutils.isEmpty(list)) {
return;
}
list.forEach(goodsVo -> {
redisTemplate.opsForValue().set("seckillGoods:" + goodsVo.getId(),goodsVo.getStockCount());
EmptyStockHap.put(goodVo.getId(),false);
});
}
@RequestMapping(value = "/doSeckill",method = RequestHethod.POST)
@ResponseBody
public RespBean doSeckill(User user,Long goodsId){
if (user == null) {
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
ValueOperations valueOperations = redisTemplate.opsForValue();
SeckillOrder seckillOrder =
(SeckillOrder) redisTemplate.opsForValue().get("order:" + user.getId() + ":" + goodsId);
if (seckillOrder != null) {
return RespBean.error(RespBeanEnum.REPEATE_ERROR);
}
if (EmptyStockHap.get(goodsId)) {
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
Long stock = valueOperations.decrement( "seckillGoods:" + goodsId);
if (stock < 0) {
EmptyStockHap.put(goodsId,true);
valueOperations.increment("seckillGoods: " + goodsId);
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
SeckillMessage seckillMessage = new SeckillMessage(user,goodsId);
mqSender.sendSeckillMessage(Jsonutil.object2JsonStr(seckillMessage));
return RespBean.success(0);
}
@RabbitListener(queues = "seckil1Queue")
public void receive(String message) {
log.info("接收的消息:" + message);
SeckillMessage seckilllessage = JsonUtil.jsonStr20bject(message,SeckillMessage.class);
Long goodId = seckillMessage.getGoodId();
User user = seckillMessage.getUser();
GoodsVo goodsVo = goodsService.findGoodsVoByGoodsId(goodId);
if (goodsVo.getStockCount() < 1) {
return;
}
SeckillOrder seckillOrder =
(SeckillOrder) redisTemplate.opsForValue().get("order:" + user.getId() + ":" + goodId);
if (seckillOrder != null) {
return;
}
orderService.seckill(user,goodsVo);
}
客户端轮询查询秒杀结果
后端
OrderController
@RequestMapping(value = "/result", method = RequestMethod.GET)
@ResponseBody
public RespBean getResult(User user,Long goodsId){
if (user == null) {
return RespBean.error(RespBeanEnum.sESSION_ERROR);
}
Long orderId = seckillOrderService.getResult(user,goodsId);
return RespBean.success(orderId);
}
OrderServiceImpl
@Transactional
@Override
public Order seckill(User user, GoodsVo goods) {
ValueOperations valueOperations = redisTemplate.opsForValue();
SeckillGoods seckillGoods = seckillGoodsService.getOne(new QueryWIrapper<SeckillGoods>().eq("goods_id",goods.getId()));
seckillGoods.setStockCount(seckillGoods.getstockCount() - 1);
boolean seckillGoodsResult = seckillGoodsService.update(new UpdateWrapper<SeckillGoods>().setSql( "stock_count = stock_count -1").eq("goods_id" , goods.getId())).gt("stock_count" , 0));
if(seckillGoods.getStockCount()<1){
valueOperations.set("isStockEmpty: "+goods.getId(),"0");
return null;
}
Order order = new Order();
order.setUserId(user.getId());
order.setGoodsId(goods.getId();
order.setDeliveryAddrId(0L);
order.setGoodsName(goods.getGoodsName());
order.setGoodsCount(1);
order.setGoodsPrice(seckillGoods.getseckillPrice());
order.set0rderChannel(1);
order.setstatus(0);
order.setCreateDate(new Date());
orderMapper.insert(order);
SeckillOrderr seckillOrder = new SeckillOrder();
seckillOrder.setuserId(user.getId());
seckillOrder.setOrderId(order. getId());
seckillOrder.setGoodsId(goods.getId());
seckillOrderService.save(seckil1Order);
redisTemplate.opsForValue().set("order:" + user.getId() + ":"+ goods.getId(), seckillOrder);
return order;
}
@Override
public Long getResult(User user,Long goodsId) {
SeckillOrder seckillOrder = seckillOrderapper.selectone(new QueryWrapper<SeckillOrder>().eq("user_id", user.getId()).eq("goods_id",goodsId));
if (null != seckillOrder) {
return seckillOrder.getorderId();
}else if (redisTemplate.hasKey("isStockEmpty: " + goodsId)) {
return -1L;
}else {
return 0L;
}
}
前端
<script>
function doSeckill() {
$.ajax({
url: '/seckill/doSeckill',
type: 'POST',
data: {
goodsId: $("#goodsId").val()
},
success: function (data){
if (data.code == 200) {
getResult($("goodsId").val());
}else {
layer.msg("客户端请求错误");
}
},
error: function () {
layer.msg("客户端请求错误");
}
})
}
function getResult(goodsId) {
g_showLoading();
$.ajax({
url: "/seckill/result",
type: "GET",
data: {
goodsId: goodsId,
},
success: function (data) {
if (data.code == 200) {
var result = data.obj;
if (result < 0) {
layer.msg("对不起,秒杀失败!");
}else if (result == 0) {
setTimeout(function () {
getResult(goodsId);
},50);
}else {
layer.confirm("恭喜你,秒杀成功!查看订单? ",{btn:["确定","取消"]},
function () {
window.location.href = "/orderDetail.html?orderId=" + result;
},
function () {
layer.close();
})
}
}
},
error: function (){
layer.msg("客户端请求错误");
}
})
}
</script>
Redis实现分布式锁
- Redis的递增递减本身带有原子性
- Redis分布式锁,锁本身是个占位的意思,当线程进来操作发现已经占位即放弃或稍候再使用,当前线程执行完毕释放锁
@Bean
public DefaultRedisscript<Boolean> script(){
DefaultRedisScript<Boolean> redisScript = new DefaultRedisScript<>();
redisScript.setLocation(new classPathResource("lock.lua"));
redisScript.setResultType(Boolean.class);
return redisscript;
}
if redis.call("get" ,KEYs[1])==ARGV[1] then
return redis.call("del", KEYS[1])
else
return 0
end
@Test
public void testLock01(){
ValueOperations valueOperations = redisTemplate.opsForValue();
Boolean isLock = valueOperations.setIfAbsent("k1","v1");
if (isLock){
ValueOperations.set("name","xxxx");
String name = (String) valueOperations.get("name");
System.out.println("name = " +name);
Integer.parseInt( "x×x×x");
redisTemplate.delete( "k1");
}else {
System.out.println("有线程在使用,请稍后再试");
}
}
@Test
public void testLock02(){
ValueOperations valueOperations = redisTemplate.opsForValue();
Boolean isLock = valueOperations.setIfAbsent("k1","v1",5,TimeUnit.SECONDS);
if (isLock){
ValueOperations.set("name","xxxx");
String name = (String) valueOperations.get("name");
System.out.println("name = " +name);
Integer.parseInt( "x×x×x");
redisTemplate.delete( "k1");
}else {
System.out.println("有线程在使用,请稍后再试");
}
}
@Test
public void testLock02(){
ValueOperations valueOperations = redisTemplate.opsForValue();
String value = UUID.randomUUID().toString();
Boolean isLock = valueOperations.setIfAbsent("k1",value,5,TimeUnit.SECONDS);
if (isLock){
ValueOperations.set("name","xxxx");
String name = (String) valueOperations.get("name");
System.out.println("name = " +name);
System.out.println(valueoperations.get("k1"));
Boolean result = (Boolean)redisTemplate.execute(script,Collections.singletonList("k1"),value);
System.out.println(result);
}else {
System.out.println("有线程请使用,请稍后");
}
}
优化Redis预减库存
采用分布式锁优化预见缓存
if (redis.call( "exists" ,KEYS[1])==1) then
local stock = tonumber(redis.call("get", KEYS[1]));
if(stock>0) then
redis.call("incrby" ,KEYS[1],-1);
return stock;
end;
return -1;
end;
@Override
public void afterPropertiesset() throws Exception {
List<GoodsVo> list = goodsservice.findGoodsVo();
if (Collectionutils.isEmpty(list)) {
return;
}
list.forEach(goodsVo -> {
redisTemplate.opsForValue().set("seckillGoods:" + goodsVo.getId(),goodsVo.getStockCount());
EmptyStockHap.put(goodVo.getId(),false);
});
}
@RequestMapping(value = "/doSeckill",method = RequestHethod.POST)
@ResponseBody
public RespBean doSeckill(User user,Long goodsId){
if (user == null) {
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
ValueOperations valueOperations = redisTemplate.opsForValue();
SeckillOrder seckillOrder =
(SeckillOrder) redisTemplate.opsForValue().get("order:" + user.getId() + ":" + goodsId);
if (seckillOrder != null) {
return RespBean.error(RespBeanEnum.REPEATE_ERROR);
}
if (EmptyStockHap.get(goodsId)) {
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
Long stock = (Long)redisTemplate.execute(script,Collections.singletonList("seckill6oods:" + goodsTd),
collections.EMPTY_LIST);
if (stock < 0) {
EmptyStockHap.put(goodsId,true);
valueOperations.increment("seckillGoods: " + goodsId);
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
SeckillMessage seckillMessage = new SeckillMessage(user,goodsId);
mqSender.sendSeckillMessage(Jsonutil.object2JsonStr(seckillMessage));
return RespBean.success(0);
}
安全优化
秒杀地址隐藏
后端
@RequestMapping(value = "/path", method = RequestHethod.GET)
@ResponseBody
public RespBean getPath(User user,Long goodsId){
if (user==null){
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
String str = orderService.createPath(user,goodsId);
return RespBean.success(str);
}
@0verride
public String createPath(User user,Long goodsId) {
String str = MD5Util.md5(UUIDUtil.uuid() + "123456");
redisTemplate.opsForValue().set("seckillPath:"+ user.getId() + ":"+ goodsId,str,60,TimeUnit.SECONDS);
return str;
}
@RequestMapping(value = "{path}/doSeckill",method = RequestHethod.POST)
@ResponseBody
public RespBean doSeckill(@PathVariable String path,User user,Long goodsId){
if (user == null) {
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
ValueOperations valueOperations = redisTemplate.opsForValue();
boolean check = orderService.checkPath(user , goodsId);
if ( !check){
return RespBean.error(RespBeanEnum.REQUEST_ILLEGAL);
}
SeckillOrder seckillOrder =
(SeckillOrder) redisTemplate.opsForValue().get("order:" + user.getId() + ":" + goodsId);
if (seckillOrder != null) {
return RespBean.error(RespBeanEnum.REPEATE_ERROR);
}
if (EmptyStockHap.get(goodsId)) {
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
Long stock = valueOperations.decrement( "seckillGoods:" + goodsId);
if (stock < 0) {
EmptyStockHap.put(goodsId,true);
valueOperations.increment("seckillGoods: " + goodsId);
return RespBean.error(RespBeanEnum.EMPTY_STOCK);
}
SeckillMessage seckillMessage = new SeckillMessage(user,goodsId);
mqSender.sendSeckillMessage(Jsonutil.object2JsonStr(seckillMessage));
return RespBean.success(0);
}
@Override
public boolean checkPath(User user,Long goodsId,String path) {
if (user == null ll goodsId <0 ll stringUtils.isEmpty(path)) {
return false;
}
String redisPath = (String) redisTemplate.opsForValue().get("seckillPath:" + user.getId() + ":"+ goodsId);
return path.equals (redisPath);
}
前端
<script>
function doSeckill(path) {
$.ajax({
url: '/seckill' + path + '/doSeckill',
type: 'POST',
data: {
goodsId: $("#goodsId").val()
},
success: function (data){
if (data.code == 200) {
getResult($("goodsId").val());
}else {
layer.msg("客户端请求错误");
}
},
error: function () {
layer.msg("客户端请求错误");
}
})
}
function getSeckillPath(){
var goodsId = $("#goodsId").val();
g_showLoading();
$.ajax({
url: "Iseckill/path",
type : "GET",
data:{
goodsId : goodsId
},
success:function (data){
if(data.code==200){
var path = data.obj;
doSeckill(path);
}else {
layer.msg(data.message);
}
},
error : function (){
layer.msg("客户端请求错误");
}
})
}
</script>
验证码
后端
<dependency>
<groupId>com.github.whvcse</groupId>
<artifactId>easy-captcha</artifactId>
<version>1.6.2</version>
</dependency>
@RequestMapping(value = "/captcha" , method = RequestHethod.GET)
public void verifyCode(User user,Long goodsId,HttpServletResponse response){
if (user==null||goodsId<0){
throw new GlobalException(RespBeanEnum.REQUEST_ILLEGAL);
}
response.setContentType("image/jpg");
response.setHeader("Pargam","No-cache");
response.setHeader("Cache-Control","no-cache");
response.setDateHeader("Expires", 0);
ArithmeticCaptcha captcha = new ArithmeticCaptcha(130,32, 3);
redisTemplate.opsForValue().set("captcha :"+user.getId()+" : "+goodsId ,captcha.text(), 300,TimeUnit.SECONDS);
try {
captcha.out(response.getoutputStream());
}catch (IOException e) {
log.error("验证码生成失败",e.getMessage());
}
}
前端
<div class="row">
<div class="form-inline">
<img id="captchaImg" width="130" height="32" onclick="refreshCaptcha()" style="display: none"/>
<input id="captcha" class="form-control" style="display: none">
<button class="btn btn-primary" type="button" id="buyButton"
onclick="getSeckillPath()">立即秒杀
<input type="hidden" name="goodsId" id="goodsId"></button>
</div>
</div>
<script>
function refreshcaptcha() {
$("#captchaImg").attr("src","/seckill/captcha?goodsId="+ $("#goodsId").val() + "&time=" + new Date();
}
</script>
@RequestMapping(value = "/path", method = RequestHethod.GET)
@ResponseBody
public RespBean getPath(User user,Long goodsId,String captcha){
if (user==null){
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
boolean check = orderService.checkCaptcha(user,goodsId,captcha);
if (!check){
return RespBean.error(RespBeanEnum.ERROR_CAPTCHA);
}
String str = orderService.createPath(user,goodsId);
return RespBean.success(str);
}
@Override
public boolean checkCaptcha(User user,Long goodsId,String captcha){
if (Stringutils.isEmpty(captcha) || user == null || goodsId < 0) {
return false;
}
String redisCaptcha = (String) redisTemplate.opsForValue().get("captcah:" + user.getId() + ":" + goodsId);
return captcha.equals(rediscaptcha);
}
接口限流
计数器算法、漏桶算法、令牌桶算法(常用)
@RequestMapping(value = "/path", method = RequestHethod.GET)
@ResponseBody
public RespBean getPath(User user,Long goodsId,String captcha,HttpServletRequest request) {
if (user==null){
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
ValueOperations valueOperations = redisTemplate.opsForValue();
String uri = request.getRequestURI();
captcha = "O";
Integer count = (Integer) valueOperations.get(uri + ":" + user.getId());
if (count == null) {
valueOperations.set(uri + ":" + user.getId(), 1,5,TimeUnit.SECONDS)
}else if (count < 5) {
valueOperations.increment(uri + ":" + user.getId());
}else {
return RespBean.error(RespBeanEnum.AcCESS_LIAIT_REAHCED);
}
boolean check = orderService.checkCaptcha(user,goodsId,captcha);
if (!check){
return RespBean.error(RespBeanEnum.ERROR_CAPTCHA);
}
String str = orderService.createPath(user,goodsId);
return RespBean.success(str);
}
以上操作冗余性大,需要进行优化
@Component
public class AccessLimitInterceptor implements HandlerInterceptor {
@Autowired
private IUserService userService;
@Override
public boolean preHandle(HttpServletRequest request,HttpServletResponse response,Object handle)throws Exception{
if (handler instanceof HandlerHethod){
User user = getUser(request, response);
UserContext.setUser(user);
IHandlerHethod hm = (HandlerHethod) handler;
AccessLimit accessLimit = hm.getHethodAnnotation(AccessLimit.class);
if (accessLimit == null){
return true;
}
int second = accessLimit.second();
int maxcount = accessLimit.maxCount();
boolean needLogin = accessLimit.needLogin();
String key = request.getRequestURI();
if (needLogin){
if (user==null){
render(response,RespBeanEnum.sESSION_ERROR);
return false;
}
key+=":"+user.getId();
}
ValueOperations valueOperations = redisTemplate.opsForValue();
Integer count = (Integer) valueOperations.get(key);
if (count == null) {
valueOperations.set(key, 1,second,TimeUnit.sECONDS);
}else if (count < maxCount) {
valueOperations.increment(key);
}else {
render(response,RespBeanEnum.ACCESS_LIMIT_REAHCED);
return false;
}
}
return true;
}
private void render(HttpServletResponse response,RespBeanEnum respBeanEnum) throws IOException {
response.setcontentType("application/json");
response.setCharacterEncoding("UTF-8");
PrintWriter out = response.getWriter();
RespBean respBean = RespBean.error(respBeanEnum);
out.write(new ObjectMapper().writeValueAsString(fespBean));
out.flush();
out.close();
}
private User getUser(HttpServletRequest request,HttpServletResponse response){
String ticket = CookieUtil.getCookievalue(request,cookieName: "userTicket");
if (Stringutils.isEmpty(ticket)) {
return null;
}
return userService.getUserByCookie(ticket, request,response);
}
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface AccessLimit {
int second();
int maxCount();
boolean needLogin() default true;
}
@AccessLimit(second=5,maxcount=5,needLogin=true)
@RequestMapping(value = "/path", method = RequestHethod.GET)
@ResponseBody
public RespBean getPath(User user,Long goodsId,String captcha,HttpServletRequest request) {
if (user==null){
return RespBean.error(RespBeanEnum.SESSION_ERROR);
}
boolean check = orderService.checkCaptcha(user,goodsId,captcha);
if (!check){
return RespBean.error(RespBeanEnum.ERROR_CAPTCHA);
}
String str = orderService.createPath(user,goodsId);
return RespBean.success(str);
}
主流秒杀方案分析
秒杀项目需要注意的点:
- 高并发以及刷接口等黑客请求对服务器端的负载冲击
- 高并发所带来的超卖问题
- 高荷载情况下下单的速度和成功率的保障
抢购之前的预约通知:点击预约产生token,token会放在用户的浏览器里,无token的用户只是在前端提示商品不足,获取token的用户可以请求后台,将重复请求前端拦截
抢购开始之前暴露接口。被黑客截取,通过脚本参与秒杀:使用网关,通过网关进行相应的限流,如:黑名单(将IP地址、用户ID),重复请求放在Redis集群,将同一个IP的发起采取拒绝考虑Redis的性能瓶颈可以做分片,带宽,统一处理
对没有token的用户:尽快处理前面已经获得token的请求,将商品进行卖光,在网关处直接终结请求,每一个Tomcat可做一到两千的QPS,令牌桶发放完就进入下单阶段
对于下单阶段要最快生成订单,否则会出现超时,可使用Redis。考虑Redis的性能可以使用分片,作用是速度快,订单查询可减少对数据库的冲击,同时订单走队列进行削峰,后端进行消费,入库成功后就可将Redis中的数据删除
出现令牌桶发放超出库存情况采用分布式锁,Redis封装好的分布式锁的方案,针对商品Id加分布式锁,但是如果商品众多,加锁反而会对性能产生影响,对Redis的压力较大
可直接在服务器实例里写好商品数量,在内存里判空,不用走Redis,不用通信,性能较高
使用到微服务采用配置中心,通过配置中心下发每个实例的商品数量,可以后台控制,在抢购开始的时候,通过配置中心下发到每个服务商品数量,当实例将内存中的商品数量消耗完毕,即为卖完了
抢购过程中服务挂掉了,大不了少卖一些,等所有服务卖完,统计订单数量,将剩余库存再次启动,再次售卖
常考问题
0、介绍一下你的项目?
为什么做这个项目?
希望将过去所学的一些知识做一个系统的深入理解。秒杀项目运用场景多,涉及的问题与中间件较为复杂,更有利于对web服务的深入学习。
详细过程?
本项目主要是为了模拟一种高并发的场景,请求到达nginx后首先经由负载轮询策略到达某一台服务器中(后端部署了两台服务器)。为了解决秒杀场景下的入口大流量、瞬时高并发问题。引入了redis作为缓存中间件,主要作用是缓存预热、预减库存等等。引入秒杀令牌与秒杀大闸机制来解决了入口大流量问题。引入线程池技术来解决了浪涌(高并发)问题。
1、秒杀中如何处理超卖问题?(网易)(百度)(美团)(滴滴)(字节)
直接由数据库操作库存的sql语句如下所示。依靠MySQL中的排他锁实现
update table_prmo set num = num - 1 WHERE id = 1001 and num > 0
利用redis 的单线程特性预减库存处理秒杀超卖问题!!!
- 在系统初始化时,将商品以及对应的库存数量预先加载到
Redis 缓存中;(缓存预热) - 接收到秒杀请求时,在
Redis 中进行预减库存(decrement),当Redis 中的库存不足时,直接返回秒杀失败,否则继续进行第3步; - 将请求放入异步队列中,返回正在排队中;
- 服务端异步队列(MQ)将请求出队,出队成功的请求可以生成秒杀订单,减少数据库库存,返回秒杀订单详情。
2、秒杀中如何解决重复下单问题?(网易)
mysql 唯一索引(商品索引)+ 分布式锁
3、热点数据失效(缓存击穿)问题如何解决?(网易)(美团)
设置热点数据永远不过期。
4、缓存和数据库数据一致性如何保证?(shopee)(美团)(网易)
5、减库存成功了,但是生成订单失败了,该怎办?(shopee)(美团)(华为)
非分布式的系统中使用Spring提供的事务功能即可。
**分布式事务:**将减库存与生成订单操作组合为一个事务。要么一起成功,要么一起失败。
CAP理论(只能保证 CP、AP)、BASE理论(最终一致性,基本可用性、柔性事务)。
分布式事务的两个协议以及几种解决方案:
- 全局消息
- 基于可靠消息(MQ)的分布式事务
- TCC
- 最大努力通知
seata 分布式事务控制组件。
6、做了什么限流削峰的措施?(字节)(美团)(华为)
秒杀令牌(token)加秒杀大闸限制入口流量。线程池技术限制瞬时并发数。验证码做防刷功能。
7、如何解决客户的恶意下单问题?(shopee)
封IP,nginx 中有一个设置,单个IP访问频率和次数多了之后有一个拉黑操作。
8、多机器扣减库存,如何保证它的线程安全的?(shopee)(美团)(华为)
分布式锁。redission 客户端实现分布式锁。
9、如何去减Redis中的库存?(华为)
decrement API减库存,increment API回增库存。以上的指令都是原子性的。
10、缓存中的数据突然失效,导致请求全部打到了数据库,如何解决?(字节)
典型的缓存雪崩问题,给缓存中的数据的过期时间加随机数。
11、如果项目中的Redis挂掉,如何减轻数据库的压力?(滴滴)(华为)
组redis 集群,主从模式、哨兵模式、集群模式。
主从模式中:如果主机宕机,使用slave of no one 断开主从关系并且把从机升级为主机。
哨兵模式中:自动监控master / slave的运行状态,基本原理是:心跳机制+投票裁决。
每个sentinel会向其它sentinel、master、slave定时发送消息(哨兵定期给主或者从和slave发送ping包(IP:port),正常则响应pong,ping和pong就叫心跳机制),以确认对方是否“活”着,如果发现对方在指定时间(可配置)内未回应,则暂时认为对方已挂(所谓的“主观认为宕机” Subjective Down,简称SDOWN)。
若master被判断死亡之后,通过选举算法,从剩下的slave节点中选一台升级为master。并自动修改相关配置。
12、页面静态化
那就把能提前放入cdn服务器的东西都放进去,反正把所有能提升效率的步骤都做一下,减少真正秒杀时候服务器的压力。
13、秒杀系统面临的问题有哪些?(滴滴)(华为)(字节)(美团)
- 高并发
- 超卖、重复卖问题
- 脚本恶意请求
- 数据库扛不住
- 加了缓存之后的缓存三大问题(击穿、穿透、雪崩)
14、秒杀系统设计?
1、nginx做一个动静分离以及负载均衡
2、redis缓存预热、预减库存
3、MQ异步下单
15、分布式会话问题?(顺丰科技)(网易)(美团)
token+redis 解决分布式会话问题。
Token是服务端生成的一串字符串,作为客户端进行请求的一个令牌,当第一次登录后,服务器生成一个userToken 便将此Token返回给客户端,存入cookie中保存,以后客户端只需带上这个userToken 前来请求数据即可,无需再次带上用户名和密码。二次登录时,只需要去redis 中获取对应token的value,验证用户信息即可。
// 用户第一次登录时,经过相关信息的验证后将对应的登录信息以及凭证(token)存入reids中
String uuid = UUID.rondom().toString();
redisTemplate.opsForValue().set(uuid, userModel);
// token下发到客户端存入cookie中进行保存
// 再次登录时cookie携带着token到redis中找到对应的value不为空,表示该用户已经登陆过了,如果查询结果为空,则让该用户重新登陆,然后将用户信息保存到redis中。
// 一般设置一个过期时间,表示的就是多久后用户的登录态就失效了。
16、线程池的执行过程?(美团)(滴滴)
先说一下核心参数:
- corePoolSize: 线程池核心线程数最大值
- maximumPoolSize: 线程池最大线程数大小
- keepAliveTime: 线程池中非核心线程空闲的存活时间大小
- unit: 线程空闲存活时间的单位
- workQueue: 存放任务的阻塞队列
- threadFactory: 用于设置创建线程的工厂,可以给创建的线程设置有意义的名字,可方便排查问题。
- handler: 线城池的饱和策略事件,主要有四种类型。
一个任务进来,先判断当前线程池中的核心线程数是否小于corePoolSize 。小于的话会直接创建一个核心线程去提交业务。如果核心线程数达到限制,那么接下来的任务会被放入阻塞队列中排队等待执行。当核心线程数达到限制且阻塞队列已满,开始创建非核心线程来执行阻塞队列中的 业务。当线程数达到了maximumPoolSize 且阻塞队列已满,那么会采用拒绝策略处理后来的业务。
17、你项目中难的难点是什么?(字节)(百度)(平安科技)(新浪)
一、限流、削峰部分的设计。
入口大流量限制
例如有10W用户来抢购10件商品,我们只放100个用户进来。
采取发放令牌机制(控制流量),根据商品id和一串uuid 产生一个令牌存入redis 中同时引入了秒杀大闸,目的是流量控制,比如当前活动商品只有100件,我们就发放500个令牌,秒杀前会先发放令牌,令牌发放完则把后来的用户挡在这一层之外,控制了流量。
获取令牌后会对比redis 中用户产生的令牌,对比成功才可以购买商品
// 设置秒杀大闸
redistemplate.opsForValue().set("door_count"+promoId, itemModel.getStock()*5)
// 发放令牌时,先去redis获取当前大闸剩余令牌数
int dazha = redistemplate.opsForValue().get("door_count"+promoId)
if (dazha <= 0) {
// 抛出一个异常
throw new exception;
}else {
String tocken = UUIDUtils.getUUID()+promoId;
// 用户只有拥有这个token才有资格下单
redistemplate.opsForValue().set(userToken, token);
}
高并发流量的限制(泄洪):利用线程池技术,维护一个具有固定线程数的线程池。每次只放固定多用户访问服务,其他用户排队。另外一种实现方式就是J.U.C 包中的信号量(Semaphore)机制。可以有效的限制线程的进入。
二、用户登录的问题(分布式会话)
做完了分布式扩展之后,发现有时候已经登录过了但是系统仍然会提示去登录,后来经过查资料发现是cookie和session的问题。然后通过设置cookie跨域分享以及利用redis 存储token信息得以解决。
18、项目中Redis都做了些什么?
- 作为缓存中间件提升系统性能
- 预减库存,防止超卖功能实现
redis 设置热点数据永不过期
19、项目中ActiveMQ都做了什么?
- 作为异步下单的中间件,利用队列排队下单缓解数据库的并发压力。
20、线程池技术中核心线程数的取值有经验值吗?(美团)(滴滴)
CPU密集型业务:N+1
IO密集型业务:2N+1
21、TPS提升了多少?(美团)
基础架构下的tps是200
经过做动静分离、nginx 反向代理并做了分布式扩展、引入redis 中间件后达到了2500 tps。
22、nginx的负载均衡策略?(字节)(顺丰科技)(大华)(跟谁学)(有赞)
轮询、权重、IP_hash、最少连接。
23、项目架构说一下?
24、引导用户去到降级页面什么意思?(字节)
25、redis缓存与mysql的数据一致性问题?(美团)
26、一个人同时用电脑和手机去抢购商品,会颁发几个token?(美团)
首先多台设备登录属于SSO问题,用户登录一端之后另外一端可以通过扫码等形式登录。虽然用户登录了多台设备,但是用户名是一样的。为用户办法的token是相同的。我们为一个用户只会颁发一个token。
27、如何利用线程池实现了流量削峰?
设置最大线程数来限制浪涌流量
28、线程池的拒绝策略能详细说一下吗?(美团)
ThreadPoolExecutor.AbortPolicy://丢弃任务并抛出RejectedExecutionException异常。
DiscardPolicy://丢弃任务,但是不抛出异常。
DiscardOldestPolicy://丢弃队列最前面的任务,然后重新提交被拒绝的任务
CallerRunsPolicy://由调用线程(提交任务的线程)处理该任务
29、被线程池拒绝掉的那部分用户的秒杀令牌还有效吗?(美团)
无效,会从redis中删除,
30、线程池中阻塞队列的大小设置为多少合适?(美团)
设置为秒杀商品的个数减去核心线程数最合适。
31、项目上线之后想看JVM的GC情况在Linux中用什么命令?(美团)
jstat -gc vmid count
jstat -gc 12538 5000
32、你做这个项目有什么预期吗?(美团)
33、秒杀令牌(token)每秒钟生成多少个?(美团)
跟随用户的请求会动态变化,令牌桶机制可以控制每秒生成令牌的个数。
34、能不能详细描述一下使用MQ异步减redis与MySQL库存的过程?(美团)
redis中库存减成功后,生成一条消息包含了商品信息、用户信息消息由MQ的生产者生产,经由queue模式发送给消费方,即订单生成的业务模块,在该模块会消费这条消息,根据其中的信息进行订单的生成,以及数据库的修改操作。
35、做到了什么程度、库存量与并发度是多少?(美团)
TPS:单机2000 QPS:
36、MySQL中的表是怎么设计的?(美团)(字节)
item表、item_stock表、order表、用户信息表、
37、假设现在你的项目需要多人协作,有没有好的办法做一个协调?(美团)(华为)
38、如何只使用MySQL保证商品没有超卖?(大华)
将查库存、减库存两个sql 语句作为一个事务进行控制,保证每一个库存只能被一个用户消费。两条语句都执行成功进行事务提交,否则回滚。但这样会导致并发很低。但也没办法。
39、数据库改库存的SQL?(美团)
update table set stock = stock-1 where prom_id = ? and stock > 1;
40、如何防止用户一直点击下单按钮?(华为)
**前端限制:**一次点击之后按钮置灰几秒钟。
**后端限制:**由于秒杀令牌的设置,用户的一个下单请求会先判断用户当前是否已经持有令牌了,因为用户全局只能获取一次令牌,然后存入到Redis缓存中。用户有令牌的话直接返回 “正在抢购中”。
|