一、背景
当我们想要某个接口访问时，优先访问redis缓存时，可以使用@Cacheable注解实现。但是在处理redis的三大问题的时候，使用自定义的注解可控性更强。根据前面redis的已知的三大问题以及它的解决方案，扩充了注解的功能。
二、实现
1.定义注解
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface RedisCheck {
????String key() default "";//key值
????int delTime() default 180;//key值删除时间
????boolean isUseNull() default true;
????String bloomFilterKey() default "";//使用的布隆过滤器的key值
????boolean isUseLock() default true;//是否使用锁
}
2.使用代理，监听注解，同时处理三大问题。
@Aspect
@Component
public class RedisCheckAop {
????@Autowired
????private RedisCacheUtil redisCacheUtil;
????@Autowired
????private BloomFilterUtil bloomFilterUtil;
????@Autowired
????private LockUtil lockUtil;
????@Around("@annotation(com.dmsl.annotation.RedisCheck)")
????public Object RedisCheck(ProceedingJoinPoint joinPoint) throws Throwable {
????????// 获取注解
????????System.out.println("----------------");
????????MethodSignature methodSignature = (MethodSignature) joinPoint.getSignature();
????????Method method = methodSignature.getMethod();
????????RedisCheck annotation = method.getAnnotation(RedisCheck.class);
????????String key = annotation.key();
????????int delTime = annotation.delTime();//设置为-1就是用不删除key 解决缓存击穿的方法二
????????boolean isUseNull = annotation.isUseNull();
????????String bloomFilterKey = annotation.bloomFilterKey();
????????boolean isUseLock = annotation.isUseLock();
????????Object[] args = joinPoint.getArgs();
????????String redisKey = KeyUtil.GetKey(key,args);
????????if(redisCacheUtil.CheckHaveCacheData(redisKey))//key存在
????????{
????????????System.out.println("key存在="+redisKey);
????????????Object data = redisCacheUtil.GetCacheData(redisKey);
????????????if(data!=null)//不为空直接返回
????????????????return data;
????????????if(isUseNull)//即使为空也返回 解决缓存穿透的方法一
????????????{
????????????????System.out.println("即使为空也返回");
????????????????return data;
????????????}
????????}
????????else//key不存在
????????{
????????????if(!bloomFilterKey.isEmpty())//使用布隆过滤器 解决缓存穿透的方法二
????????????{
????????????????BloomFilter bloomFilter=bloomFilterUtil.GetBloomFilter(bloomFilterKey);
????????????????if(bloomFilter!=null)//布隆过滤器存在
????????????????{
????????????????????System.out.println("布隆过滤器存在");
????????????????????//不存在说明请求的redisKey mysql不存在
????????????????????if(!bloomFilter.contains(redisKey)) {
????????????????????????System.out.println("redisKey mysql不存在");
????????????????????????return null;
????????????????????}
????????????????}
????????????}
????????????System.out.println("key不存在="+redisKey);
????????????if(isUseLock)//使用互斥锁 解决缓存击穿和雪崩的方法二
????????????{
????????????????if(lockUtil.CheckIsLock(redisKey))//被锁了
????????????????{
????????????????????System.out.println("已经锁了");
????????????????????Thread.sleep(100);
????????????????????return RedisCheck(joinPoint);
????????????????}
????????????????else{
????????????????????System.out.println("没锁，现在加上锁了");
????????????????}
????????????}
????????}
????????Object result = null;
????????try{
????????????result = joinPoint.proceed(args);//调用原本的Service函数，访问mysql获取数据
????????????redisCacheUtil.AddCacheData(redisKey,result,delTime);
????????????System.out.println("请求mysql");
????????}
????????finally {
????????????if(isUseLock)
????????????{
????????????????System.out.println("释放锁");
????????????????lockUtil.UnLock(redisKey);//防止报错
????????????}
????????}
????????return result;
????}
}
这里的解决方法并非最优，一般需要根据项目做修改。
例如：
解决缓存击穿的问题中，这里我们是存下空值，避免下一次请求不存在的数据又访问数据库。但是恶意用户攻击可能会换不同的不存在的key来攻击（例如：我key值选-1递减的来请求），这样会导致内存不断的增加，因此加入了布隆过滤器来处理。
但是如果已知一个项目中的表key值id是大于0，小于N（这个N是可以获取到的）。并且0-N之前被删的id不多，那么我完全可以不用布隆过滤器，直接判断id是不是在0-N之外，就可以确定id是不是不存在的了。就算是最坏的情况，redis也就只是多了0-N之间被删id的空数据。
3.Redis缓存管理工具
@Component
public class RedisCacheUtil {
????@Resource
????private RedisUtil redisUtil;
????public Object GetCacheData(String key, Object[] args )
????{
????????key = GetKey(key,args);
????????return GetCacheData(key);
????}
????public Object GetCacheData(String redisKey)
????{
????????Object data =null;
????????if (redisUtil.hasKey(redisKey)) {
????????????data = redisUtil.get(redisKey);
????????}
????????return data;
????}
????public boolean CheckHaveCacheData(String key, Object[] args )
????{
????????key = GetKey(key,args);
????????return CheckHaveCacheData(key);
????}
????public boolean CheckHaveCacheData(String redisKey)
????{
????????return redisUtil.hasKey(redisKey);
????}
????public void AddCacheData(String key, Object[] args,Object data,int delTime)
????{
????????key = GetKey(key,args);
????????AddCacheData(key,data,delTime);
????}
????public void AddCacheData(String redisKey, Object data,int delTime)
????{
????????redisUtil.set(redisKey, data);
????????if(delTime>0)
????????????redisUtil.expire(redisKey, delTime);
????}
????public void RemoveCacheData(String key, Object[] args){
????????key = GetKey(key,args);
????????if (redisUtil.hasKey(key)) {
????????????redisUtil.del(key);
????????}
????}
????public void RemoveCacheData(String key, Object arg){
????????key = GetKey(key,arg);
????????if (redisUtil.hasKey(key)) {
????????????redisUtil.del(key);
????????}
????}
}
4.布隆过滤器实现
public class BloomFilter {
????private static final int BIT_SIZE = 12;
????private static final int DEFAULT_SIZE = 2 << BIT_SIZE;
????private static final int[] SEEDS = new int[]{3, 13, 46};
????private BitSet bits = new BitSet(DEFAULT_SIZE);
????private HashCode[] hashFunc = new HashCode[SEEDS.length];
????public BloomFilter() {
????????for (int i = 0; i < SEEDS.length; i++) {
????????????hashFunc[i] = new HashCode(DEFAULT_SIZE, SEEDS[i]);
????????}
????}
????
????public void add(Object value) {
????????for (HashCode f : hashFunc) {
????????????bits.set(f.hash(value), true);
????????}
????}
????public boolean contains(Object value) {
????????boolean ret = true;
????????for (HashCode f : hashFunc) {
????????????ret = ret && bits.get(f.hash(value));
????????}
????????return ret;
????}
????
????public static class HashCode {
????????private int bitSize;
????????private int seed;
????????public HashCode(int bitSize, int seed) {
????????????this.bitSize = bitSize;
????????????this.seed = seed;
????????}
????????public int hash(Object value) {
????????????int h;
????????????return (value == null) ? 0 : Math.abs(seed * (bitSize - 1) & ((h = value.hashCode()) ^ (h >>> 16)));
????????}
????}
}
5.布隆过滤器管理
@Component
public class BloomFilterUtil {
????private Map<String,BloomFilter> bloomFilterMap=new HashMap<>();
????public BloomFilter GetBloomFilter(String key)
????{
????????if(!bloomFilterMap.containsKey(key))
????????????return null;
????????return bloomFilterMap.get(key);
????}
????public void AddBloomFilter(String key,BloomFilter bloomFilter)
????{
????????bloomFilterMap.put(key,bloomFilter);
????}
}
6.互斥锁
@Component
public class LockUtil {
????private static final String lockKey="lockKey";
????private Set<String> lockSet=new CopyOnWriteArraySet<>();
????public boolean CheckIsLock(String key){
????????key = GetKey(lockKey,key);
???????return !lockSet.add(key);
????}
????public void UnLock(String key){
????????key = GetKey(lockKey,key);
????????lockSet.remove(key);
????}
}
7.使用方法
@RedisCheck(key = RedisKey.topViewVideoKey)
public List<VideoEntity> FindTopViewCountVideo(int start, int len) {
????return videoMapper.FindTopViewCountVideo(start, len);
}
8.服务器启动时，初始化布隆过滤器
public void InitBloomFilter(){
????List<VideoEntity> videoEntityList = videoService.GetAllVideo();
????BloomFilter bloomFilter=new BloomFilter();
????for(VideoEntity videoEntity:videoEntityList)
????{
????????bloomFilter.add(KeyUtil.GetKey(RedisKey.videoKey,videoEntity.getId()));
????}
????bloomFilterUtil.AddBloomFilter(BloomFilterKey.videoKey,bloomFilter);
}
三、测试用例
一、测试布隆过滤器：
设置
@RedisCheck(key= RedisKey.videoKey,isUseNull=false,bloomFilterKey= BloomFilterKey.videoKey)
单元测试
@Test
//10个线程 执行10次
//@PerfTest(invocations = 10,threads = 10)
public void Test(){
????appInitService.InitBloomFilter();
????VideoEntity videoEntity= videoService.FindVideoByID(-1);
????System.out.println("+++++++++null="+(videoEntity==null));
}
1. 输入不存在的key=-1请求一次。预期输出：
----------------
布隆过滤器存在
redisKey mysql不存在
+++++++++null=true
2.输入不存在的key=-1再请求一次。预期输出和上面一致。
二、测试是否为空也返回
设置
@RedisCheck(key= RedisKey.videoKey,isUseNull=true)
单元测试
@Test
//10个线程 执行10次
//@PerfTest(invocations = 10,threads = 10)
public void Test(){
????appInitService.InitBloomFilter();
????VideoEntity videoEntity= videoService.FindVideoByID(-1);
????System.out.println("+++++++++null="+(videoEntity==null));
}
1. 输入不存在的key=-2请求一次。预期输出：
----------------
key不存在=video--2
没锁，现在加上锁了
请求mysql
释放锁
+++++++++null=true
2. 输入不存在的key=-2再请求一次。预期输出：
----------------
key存在=video--2
即使为空也返回
+++++++++null=true
三、测试互斥锁
设置
@RedisCheck(key= RedisKey.videoKey,isUseLock=true)
单元测试
开10个线程，总共执行10次
@Rule
public ContiPerfRule contiPerfRule = new ContiPerfRule();
@Test
//10个线程 执行10次
@PerfTest(invocations = 10,threads = 10)
public void Test(){
????appInitService.InitBloomFilter();
????VideoEntity videoEntity= videoService.FindVideoByID(152);
????System.out.println("+++++++++");
}
pom.xml需要引入
<dependency>
????<groupId>org.databene</groupId>
????<artifactId>contiperf</artifactId>
????<version>2.3.4</version>
????<scope>test</scope>
</dependency>
1. 输入存在的key=156执行。
当输出中，打印了两次以上“key不存在”（打印一次不能说错，但是达不到测试目的），“请求mysql”打印了一次，那就是正常的。
----------------
key不存在=video-156
没锁，现在加上锁了
----------------
key不存在=video-156
已经锁了
----------------
请求mysql
释放锁
+++++++++null=false
key存在=video-156
----------------
----------------
key存在=video-156
----------------
----------------
----------------
key存在=video-156
key存在=video-156
key存在=video-156
key存在=video-156
----------------
+++++++++null=false
----------------
key存在=video-156
+++++++++null=false
key存在=video-156
+++++++++null=false
+++++++++null=false
+++++++++null=false
+++++++++null=false
+++++++++null=false
+++++++++null=false
----------------
key存在=video-156
+++++++++null=false
|